2024-09-10T18:06:00.530 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T18:06:00.546 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T18:06:00.622 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900210 branch: wip-guits-quincy-2024-09-09-1810-quincy description: orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rotate-keys} email: gabrioux@ibm.com first_in_suite: false job_id: '7900210' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-quincy-2024-09-09-1810-quincy ceph: conf: global: mon election default strategy: 1 ms bind msgr1: false ms bind msgr2: true ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 5be57395ea235535910512ae73ba8e7ff25fe588 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: cephadm-package install: ceph: flavor: default sha1: 5be57395ea235535910512ae73ba8e7ff25fe588 extra_packages: - cephadm selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: quincy-orch-centos-9-teuth sha1: fad5356a94e6044ce1062d748daa5a0f2ed53a7d owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 4625 sha1: 5be57395ea235535910512ae73ba8e7ff25fe588 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: quincy-orch-centos-9-teuth suite_path: /home/teuthworker/src/github.com_adk3798_ceph_fad5356a94e6044ce1062d748daa5a0f2ed53a7d/qa suite_relpath: qa suite_repo: https://github.com/adk3798/ceph.git suite_sha1: fad5356a94e6044ce1062d748daa5a0f2ed53a7d targets: smithi115.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP6EZFzdG0IBmETDgbaLm7UU7x2JFAh/o+/sPTW5eGaoJCe87kkS/d7CRSrkuBufaP7H6EtMi7BpSxdy6x5I7vk= smithi150.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNgoT4+brAiYyFVZfv9tI3Oy+QhNynKKg68hYAtZIPCQyYMVbXjd8S5l9QGAopzP5WLSVaY8L7Pg23FL3u+zGtk= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install runc nvmetcli nvme-cli -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - cephadm.shell: mon.a: - "set -ex\nfor f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x\n\ do\n echo \"rotating key for $f\"\n K=$(ceph auth get-key $f)\n NK=\"\ $K\"\n ceph orch daemon rotate-key $f\n while [ \"$K\" == \"$NK\" ]; do\n\ \ sleep 5\n NK=$(ceph auth get-key $f)\n done\ndone\n" teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_17:53:37 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T18:06:00.622 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_adk3798_ceph_fad5356a94e6044ce1062d748daa5a0f2ed53a7d/qa; will attempt to use it 2024-09-10T18:06:00.623 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_adk3798_ceph_fad5356a94e6044ce1062d748daa5a0f2ed53a7d/qa/tasks 2024-09-10T18:06:00.623 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T18:06:00.624 INFO:teuthology.task.internal:Checking packages... 2024-09-10T18:06:00.645 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '5be57395ea235535910512ae73ba8e7ff25fe588' 2024-09-10T18:06:00.646 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T18:06:00.646 INFO:teuthology.packaging:ref: None 2024-09-10T18:06:00.646 INFO:teuthology.packaging:tag: None 2024-09-10T18:06:00.646 INFO:teuthology.packaging:branch: wip-guits-quincy-2024-09-09-1810-quincy 2024-09-10T18:06:00.646 INFO:teuthology.packaging:sha1: 5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:06:00.646 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-quincy-2024-09-09-1810-quincy 2024-09-10T18:06:00.788 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1606.g5be57395 2024-09-10T18:06:00.789 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T18:06:00.795 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T18:06:00.795 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T18:06:00.829 INFO:teuthology.task.internal:Saving configuration 2024-09-10T18:06:00.840 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T18:06:00.864 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T18:06:00.887 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi115.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900210', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 18:00:47.727551', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP6EZFzdG0IBmETDgbaLm7UU7x2JFAh/o+/sPTW5eGaoJCe87kkS/d7CRSrkuBufaP7H6EtMi7BpSxdy6x5I7vk='} 2024-09-10T18:06:00.909 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi150.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900210', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 18:00:47.726182', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNgoT4+brAiYyFVZfv9tI3Oy+QhNynKKg68hYAtZIPCQyYMVbXjd8S5l9QGAopzP5WLSVaY8L7Pg23FL3u+zGtk='} 2024-09-10T18:06:00.909 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T18:06:00.915 INFO:teuthology.task.internal:roles: ubuntu@smithi115.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-10T18:06:00.915 INFO:teuthology.task.internal:roles: ubuntu@smithi150.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-09-10T18:06:00.915 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T18:06:00.992 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f33a716bac0>, signals=[15]) 2024-09-10T18:06:00.992 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T18:06:00.998 INFO:teuthology.task.internal:Opening connections... 2024-09-10T18:06:00.998 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi115.front.sepia.ceph.com 2024-09-10T18:06:00.999 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:06:01.074 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi150.front.sepia.ceph.com 2024-09-10T18:06:01.075 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:06:01.150 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T18:06:01.156 DEBUG:teuthology.orchestra.run.smithi115:> uname -m 2024-09-10T18:06:01.175 INFO:teuthology.orchestra.run.smithi115.stdout:x86_64 2024-09-10T18:06:01.175 DEBUG:teuthology.orchestra.run.smithi115:> cat /etc/os-release 2024-09-10T18:06:01.234 INFO:teuthology.orchestra.run.smithi115.stdout:NAME="CentOS Stream" 2024-09-10T18:06:01.234 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION="9" 2024-09-10T18:06:01.234 INFO:teuthology.orchestra.run.smithi115.stdout:ID="centos" 2024-09-10T18:06:01.234 INFO:teuthology.orchestra.run.smithi115.stdout:ID_LIKE="rhel fedora" 2024-09-10T18:06:01.234 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION_ID="9" 2024-09-10T18:06:01.234 INFO:teuthology.orchestra.run.smithi115.stdout:PLATFORM_ID="platform:el9" 2024-09-10T18:06:01.234 INFO:teuthology.orchestra.run.smithi115.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T18:06:01.234 INFO:teuthology.orchestra.run.smithi115.stdout:ANSI_COLOR="0;31" 2024-09-10T18:06:01.234 INFO:teuthology.orchestra.run.smithi115.stdout:LOGO="fedora-logo-icon" 2024-09-10T18:06:01.234 INFO:teuthology.orchestra.run.smithi115.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T18:06:01.235 INFO:teuthology.orchestra.run.smithi115.stdout:HOME_URL="https://centos.org/" 2024-09-10T18:06:01.235 INFO:teuthology.orchestra.run.smithi115.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T18:06:01.235 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T18:06:01.235 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T18:06:01.236 INFO:teuthology.lock.ops:Updating smithi115.front.sepia.ceph.com on lock server 2024-09-10T18:06:01.249 DEBUG:teuthology.orchestra.run.smithi150:> uname -m 2024-09-10T18:06:01.268 INFO:teuthology.orchestra.run.smithi150.stdout:x86_64 2024-09-10T18:06:01.268 DEBUG:teuthology.orchestra.run.smithi150:> cat /etc/os-release 2024-09-10T18:06:01.327 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="CentOS Stream" 2024-09-10T18:06:01.327 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="9" 2024-09-10T18:06:01.327 INFO:teuthology.orchestra.run.smithi150.stdout:ID="centos" 2024-09-10T18:06:01.327 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE="rhel fedora" 2024-09-10T18:06:01.327 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="9" 2024-09-10T18:06:01.327 INFO:teuthology.orchestra.run.smithi150.stdout:PLATFORM_ID="platform:el9" 2024-09-10T18:06:01.327 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T18:06:01.328 INFO:teuthology.orchestra.run.smithi150.stdout:ANSI_COLOR="0;31" 2024-09-10T18:06:01.328 INFO:teuthology.orchestra.run.smithi150.stdout:LOGO="fedora-logo-icon" 2024-09-10T18:06:01.328 INFO:teuthology.orchestra.run.smithi150.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T18:06:01.328 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://centos.org/" 2024-09-10T18:06:01.328 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T18:06:01.328 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T18:06:01.328 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T18:06:01.328 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2024-09-10T18:06:01.349 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T18:06:01.358 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T18:06:01.422 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T18:06:01.422 DEBUG:teuthology.orchestra.run.smithi115:> test '!' -e /home/ubuntu/cephtest 2024-09-10T18:06:01.425 DEBUG:teuthology.orchestra.run.smithi150:> test '!' -e /home/ubuntu/cephtest 2024-09-10T18:06:01.442 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T18:06:01.447 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T18:06:01.448 DEBUG:teuthology.orchestra.run.smithi115:> test -z $(ls -A /var/lib/ceph) 2024-09-10T18:06:01.482 DEBUG:teuthology.orchestra.run.smithi150:> test -z $(ls -A /var/lib/ceph) 2024-09-10T18:06:01.505 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T18:06:01.548 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T18:06:01.562 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-10T18:06:01.562 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-10T18:06:01.563 DEBUG:teuthology.orchestra.run.smithi115:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T18:06:01.563 DEBUG:teuthology.orchestra.run.smithi150:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T18:06:01.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:06:01.580 DEBUG:teuthology.orchestra.run.smithi150:> uname -r 2024-09-10T18:06:01.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:06:01.581 DEBUG:teuthology.orchestra.run.smithi115:> uname -r 2024-09-10T18:06:01.637 INFO:teuthology.orchestra.run.smithi150.stdout:5.14.0-503.el9.x86_64 2024-09-10T18:06:01.638 INFO:teuthology.task.kernel:Running kernel on smithi150: 5.14.0-503.el9.x86_64 2024-09-10T18:06:01.638 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum install -y kernel 2024-09-10T18:06:01.639 INFO:teuthology.orchestra.run.smithi115.stdout:5.14.0-503.el9.x86_64 2024-09-10T18:06:01.639 INFO:teuthology.task.kernel:Running kernel on smithi115: 5.14.0-503.el9.x86_64 2024-09-10T18:06:01.640 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum install -y kernel 2024-09-10T18:06:05.152 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.2 MB 00:01 2024-09-10T18:06:05.631 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - BaseOS 3.7 MB/s | 8.2 MB 00:02 2024-09-10T18:06:07.674 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - AppStream 23 MB/s | 20 MB 00:00 2024-09-10T18:06:08.953 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - AppStream 7.6 MB/s | 20 MB 00:02 2024-09-10T18:06:13.270 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-10T18:06:13.524 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - CRB 3.2 MB/s | 6.5 MB 00:02 2024-09-10T18:06:14.796 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - Extras packages 58 kB/s | 19 kB 00:00 2024-09-10T18:06:15.108 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 19 kB 00:00 2024-09-10T18:06:15.792 INFO:teuthology.orchestra.run.smithi150.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-10T18:06:15.953 INFO:teuthology.orchestra.run.smithi115.stdout:Extra Packages for Enterprise Linux 31 MB/s | 23 MB 00:00 2024-09-10T18:06:21.206 INFO:teuthology.orchestra.run.smithi150.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-10T18:06:21.451 INFO:teuthology.orchestra.run.smithi115.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-09-10T18:06:23.229 INFO:teuthology.orchestra.run.smithi150.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T18:06:23.229 INFO:teuthology.orchestra.run.smithi150.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T18:06:23.317 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T18:06:23.325 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:06:23.325 INFO:teuthology.orchestra.run.smithi150.stdout: Package Architecture Version Repository Size 2024-09-10T18:06:23.326 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:06:23.326 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-10T18:06:23.326 INFO:teuthology.orchestra.run.smithi150.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T18:06:23.326 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-09-10T18:06:23.326 INFO:teuthology.orchestra.run.smithi150.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T18:06:23.326 INFO:teuthology.orchestra.run.smithi150.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T18:06:23.326 INFO:teuthology.orchestra.run.smithi150.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T18:06:23.326 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:06:23.326 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T18:06:23.326 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:06:23.326 INFO:teuthology.orchestra.run.smithi150.stdout:Install 4 Packages 2024-09-10T18:06:23.326 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:06:23.327 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 79 M 2024-09-10T18:06:23.327 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 126 M 2024-09-10T18:06:23.327 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-10T18:06:23.490 INFO:teuthology.orchestra.run.smithi115.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T18:06:23.490 INFO:teuthology.orchestra.run.smithi115.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T18:06:23.580 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T18:06:23.584 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:06:23.584 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2024-09-10T18:06:23.584 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:06:23.584 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T18:06:23.584 INFO:teuthology.orchestra.run.smithi115.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T18:06:23.584 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-10T18:06:23.584 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T18:06:23.584 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T18:06:23.584 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T18:06:23.584 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:06:23.585 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T18:06:23.585 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:06:23.585 INFO:teuthology.orchestra.run.smithi115.stdout:Install 4 Packages 2024-09-10T18:06:23.585 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:06:23.585 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 79 M 2024-09-10T18:06:23.586 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 126 M 2024-09-10T18:06:23.586 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T18:06:23.716 INFO:teuthology.orchestra.run.smithi150.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 79 kB/s | 23 kB 00:00 2024-09-10T18:06:23.827 INFO:teuthology.orchestra.run.smithi115.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 185 kB/s | 23 kB 00:00 2024-09-10T18:06:25.560 INFO:teuthology.orchestra.run.smithi115.stdout:(2/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 19 MB/s | 35 MB 00:01 2024-09-10T18:06:30.509 INFO:teuthology.orchestra.run.smithi115.stdout:(3/4): kernel-core-5.14.0-505.el9.x86_64.rpm 2.3 MB/s | 16 MB 00:06 2024-09-10T18:06:30.886 INFO:teuthology.orchestra.run.smithi115.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 4.0 MB/s | 29 MB 00:07 2024-09-10T18:06:30.886 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:06:30.887 INFO:teuthology.orchestra.run.smithi115.stdout:Total 11 MB/s | 79 MB 00:07 2024-09-10T18:06:31.565 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T18:06:31.905 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T18:06:31.905 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T18:06:31.907 INFO:teuthology.orchestra.run.smithi150.stdout:(2/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 4.1 MB/s | 35 MB 00:08 2024-09-10T18:06:32.165 INFO:teuthology.orchestra.run.smithi150.stdout:(3/4): kernel-core-5.14.0-505.el9.x86_64.rpm 1.8 MB/s | 16 MB 00:08 2024-09-10T18:06:32.898 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T18:06:32.898 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T18:06:34.165 INFO:teuthology.orchestra.run.smithi150.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 2.7 MB/s | 29 MB 00:10 2024-09-10T18:06:34.166 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:06:34.166 INFO:teuthology.orchestra.run.smithi150.stdout:Total 7.3 MB/s | 79 MB 00:10 2024-09-10T18:06:34.843 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T18:06:35.212 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T18:06:35.212 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T18:06:35.337 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T18:06:36.053 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T18:06:36.143 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:06:36.168 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T18:06:36.168 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T18:06:38.626 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T18:06:38.946 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:06:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:06:39.266 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T18:06:39.466 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:06:42.259 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:06:42.452 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:06:43.945 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:06:44.196 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:06:47.259 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:06:47.295 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:06:49.059 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:06:52.158 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:07:27.282 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:07:27.322 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:07:27.722 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:07:27.723 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T18:07:27.723 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:07:27.723 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:07:28.595 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:07:28.595 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:07:28.595 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T18:07:28.595 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T18:07:28.595 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T18:07:28.595 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T18:07:28.595 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T18:07:28.595 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:07:28.596 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T18:07:28.930 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T18:07:29.009 INFO:teuthology.orchestra.run.smithi115.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T18:07:29.009 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T18:07:29.010 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T18:07:29.010 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T18:07:29.010 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T18:07:29.010 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-10T18:07:29.010 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi115.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T18:07:29.010 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T18:07:29.069 INFO:teuthology.orchestra.run.smithi115.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T18:07:29.069 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T18:07:29.069 DEBUG:teuthology.orchestra.run.smithi115:> sudo rpm -qi grub2-tools 2024-09-10T18:07:29.103 INFO:teuthology.orchestra.run.smithi115.stdout:Name : grub2-tools 2024-09-10T18:07:29.103 INFO:teuthology.orchestra.run.smithi115.stdout:Epoch : 1 2024-09-10T18:07:29.103 INFO:teuthology.orchestra.run.smithi115.stdout:Version : 2.06 2024-09-10T18:07:29.103 INFO:teuthology.orchestra.run.smithi115.stdout:Release : 82.el9 2024-09-10T18:07:29.103 INFO:teuthology.orchestra.run.smithi115.stdout:Architecture: x86_64 2024-09-10T18:07:29.103 INFO:teuthology.orchestra.run.smithi115.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T18:07:29.103 INFO:teuthology.orchestra.run.smithi115.stdout:Group : Unspecified 2024-09-10T18:07:29.103 INFO:teuthology.orchestra.run.smithi115.stdout:Size : 8274670 2024-09-10T18:07:29.103 INFO:teuthology.orchestra.run.smithi115.stdout:License : GPLv3+ 2024-09-10T18:07:29.103 INFO:teuthology.orchestra.run.smithi115.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T18:07:29.104 INFO:teuthology.orchestra.run.smithi115.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T18:07:29.104 INFO:teuthology.orchestra.run.smithi115.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T18:07:29.104 INFO:teuthology.orchestra.run.smithi115.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T18:07:29.104 INFO:teuthology.orchestra.run.smithi115.stdout:Packager : builder@centos.org 2024-09-10T18:07:29.104 INFO:teuthology.orchestra.run.smithi115.stdout:Vendor : CentOS 2024-09-10T18:07:29.104 INFO:teuthology.orchestra.run.smithi115.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T18:07:29.104 INFO:teuthology.orchestra.run.smithi115.stdout:Summary : Support tools for GRUB. 2024-09-10T18:07:29.104 INFO:teuthology.orchestra.run.smithi115.stdout:Description : 2024-09-10T18:07:29.104 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:07:29.104 INFO:teuthology.orchestra.run.smithi115.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T18:07:29.104 INFO:teuthology.orchestra.run.smithi115.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T18:07:29.104 INFO:teuthology.orchestra.run.smithi115.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T18:07:29.104 INFO:teuthology.orchestra.run.smithi115.stdout:hardware devices. 2024-09-10T18:07:29.105 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:07:29.105 INFO:teuthology.orchestra.run.smithi115.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T18:07:29.105 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T18:07:29.106 INFO:teuthology.task.kernel:Updating grub on smithi115 to boot 5.14.0-505.el9.x86_64 2024-09-10T18:07:29.106 DEBUG:teuthology.orchestra.run.smithi115:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T18:07:29.782 INFO:teuthology.orchestra.run.smithi115.stderr:Generating grub configuration file ... 2024-09-10T18:07:30.657 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:07:30.700 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:07:31.102 INFO:teuthology.orchestra.run.smithi115.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T18:07:31.112 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:07:31.112 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T18:07:31.112 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:07:31.112 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:07:31.133 INFO:teuthology.orchestra.run.smithi115.stderr:done 2024-09-10T18:07:31.136 DEBUG:teuthology.orchestra.run.smithi115:> mktemp 2024-09-10T18:07:31.152 INFO:teuthology.orchestra.run.smithi115.stdout:/tmp/tmp.W8QFTflhdB 2024-09-10T18:07:31.152 DEBUG:teuthology.orchestra.run.smithi115:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.W8QFTflhdB 2024-09-10T18:07:31.219 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0666 /tmp/tmp.W8QFTflhdB 2024-09-10T18:07:31.356 DEBUG:teuthology.orchestra.remote:smithi115:/tmp/tmp.W8QFTflhdB is 6KB 2024-09-10T18:07:31.404 DEBUG:teuthology.orchestra.run.smithi115:> rm -fr /tmp/tmp.W8QFTflhdB 2024-09-10T18:07:31.419 DEBUG:teuthology.orchestra.run.smithi115:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T18:07:31.491 INFO:teuthology.orchestra.run.smithi115.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T18:07:31.491 INFO:teuthology.orchestra.run.smithi115.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T18:07:31.491 INFO:teuthology.orchestra.run.smithi115.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T18:07:31.491 INFO:teuthology.orchestra.run.smithi115.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T18:07:31.492 DEBUG:teuthology.orchestra.run.smithi115:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T18:07:31.659 DEBUG:teuthology.orchestra.run.smithi115:> sudo shutdown -r now 2024-09-10T18:07:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:07:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:07:31.980 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-10T18:07:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T18:07:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T18:07:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T18:07:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T18:07:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:07:31.980 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T18:07:32.313 DEBUG:teuthology.orchestra.run.smithi150:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T18:07:32.438 INFO:teuthology.orchestra.run.smithi150.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T18:07:32.438 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T18:07:32.438 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T18:07:32.438 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T18:07:32.438 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T18:07:32.438 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-10T18:07:32.439 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi150.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T18:07:32.439 DEBUG:teuthology.orchestra.run.smithi150:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T18:07:32.497 INFO:teuthology.orchestra.run.smithi150.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T18:07:32.497 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T18:07:32.497 DEBUG:teuthology.orchestra.run.smithi150:> sudo rpm -qi grub2-tools 2024-09-10T18:07:32.538 INFO:teuthology.orchestra.run.smithi150.stdout:Name : grub2-tools 2024-09-10T18:07:32.538 INFO:teuthology.orchestra.run.smithi150.stdout:Epoch : 1 2024-09-10T18:07:32.538 INFO:teuthology.orchestra.run.smithi150.stdout:Version : 2.06 2024-09-10T18:07:32.538 INFO:teuthology.orchestra.run.smithi150.stdout:Release : 82.el9 2024-09-10T18:07:32.538 INFO:teuthology.orchestra.run.smithi150.stdout:Architecture: x86_64 2024-09-10T18:07:32.538 INFO:teuthology.orchestra.run.smithi150.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T18:07:32.538 INFO:teuthology.orchestra.run.smithi150.stdout:Group : Unspecified 2024-09-10T18:07:32.539 INFO:teuthology.orchestra.run.smithi150.stdout:Size : 8274670 2024-09-10T18:07:32.539 INFO:teuthology.orchestra.run.smithi150.stdout:License : GPLv3+ 2024-09-10T18:07:32.539 INFO:teuthology.orchestra.run.smithi150.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T18:07:32.539 INFO:teuthology.orchestra.run.smithi150.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T18:07:32.539 INFO:teuthology.orchestra.run.smithi150.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T18:07:32.539 INFO:teuthology.orchestra.run.smithi150.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T18:07:32.539 INFO:teuthology.orchestra.run.smithi150.stdout:Packager : builder@centos.org 2024-09-10T18:07:32.539 INFO:teuthology.orchestra.run.smithi150.stdout:Vendor : CentOS 2024-09-10T18:07:32.539 INFO:teuthology.orchestra.run.smithi150.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T18:07:32.539 INFO:teuthology.orchestra.run.smithi150.stdout:Summary : Support tools for GRUB. 2024-09-10T18:07:32.539 INFO:teuthology.orchestra.run.smithi150.stdout:Description : 2024-09-10T18:07:32.539 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:07:32.539 INFO:teuthology.orchestra.run.smithi150.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T18:07:32.539 INFO:teuthology.orchestra.run.smithi150.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T18:07:32.540 INFO:teuthology.orchestra.run.smithi150.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T18:07:32.540 INFO:teuthology.orchestra.run.smithi150.stdout:hardware devices. 2024-09-10T18:07:32.540 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:07:32.540 INFO:teuthology.orchestra.run.smithi150.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T18:07:32.541 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T18:07:32.541 INFO:teuthology.task.kernel:Updating grub on smithi150 to boot 5.14.0-505.el9.x86_64 2024-09-10T18:07:32.541 DEBUG:teuthology.orchestra.run.smithi150:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T18:07:33.234 INFO:teuthology.orchestra.run.smithi150.stderr:Generating grub configuration file ... 2024-09-10T18:07:34.522 INFO:teuthology.orchestra.run.smithi150.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T18:07:34.549 INFO:teuthology.orchestra.run.smithi150.stderr:done 2024-09-10T18:07:34.551 DEBUG:teuthology.orchestra.run.smithi150:> mktemp 2024-09-10T18:07:34.567 INFO:teuthology.orchestra.run.smithi150.stdout:/tmp/tmp.kqj7qHx3lf 2024-09-10T18:07:34.567 DEBUG:teuthology.orchestra.run.smithi150:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.kqj7qHx3lf 2024-09-10T18:07:34.637 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0666 /tmp/tmp.kqj7qHx3lf 2024-09-10T18:07:34.774 DEBUG:teuthology.orchestra.remote:smithi150:/tmp/tmp.kqj7qHx3lf is 6KB 2024-09-10T18:07:34.822 DEBUG:teuthology.orchestra.run.smithi150:> rm -fr /tmp/tmp.kqj7qHx3lf 2024-09-10T18:07:34.837 DEBUG:teuthology.orchestra.run.smithi150:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T18:07:34.900 INFO:teuthology.orchestra.run.smithi150.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T18:07:34.900 INFO:teuthology.orchestra.run.smithi150.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T18:07:34.900 INFO:teuthology.orchestra.run.smithi150.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T18:07:34.900 INFO:teuthology.orchestra.run.smithi150.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T18:07:34.901 DEBUG:teuthology.orchestra.run.smithi150:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T18:07:35.047 DEBUG:teuthology.orchestra.run.smithi150:> sudo shutdown -r now 2024-09-10T18:08:01.685 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-10T18:08:01.686 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-10T18:08:01.687 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:08:05.053 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-10T18:08:05.053 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-10T18:08:05.053 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:08:36.161 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.115 2024-09-10T18:08:39.741 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.150 2024-09-10T18:08:45.165 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-10T18:08:45.166 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:08:48.745 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-10T18:08:48.746 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:08:51.809 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.150 2024-09-10T18:09:03.821 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-10T18:09:03.822 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:09:06.881 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.150 2024-09-10T18:09:16.903 DEBUG:teuthology.orchestra.run.smithi115:> true 2024-09-10T18:09:17.336 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-10T18:09:17.336 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-10T18:09:17.337 DEBUG:teuthology.orchestra.run.smithi115:> uname -r 2024-09-10T18:09:17.353 INFO:teuthology.orchestra.run.smithi115.stdout:5.14.0-505.el9.x86_64 2024-09-10T18:09:17.353 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T18:09:17.353 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T18:09:17.354 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T18:09:18.354 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-10T18:09:18.354 DEBUG:teuthology.orchestra.run.smithi115:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T18:09:18.450 INFO:teuthology.orchestra.run.smithi115.stdout:ttyS1 2024-09-10T18:09:18.469 DEBUG:teuthology.parallel:result is None 2024-09-10T18:09:21.882 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-10T18:09:21.883 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:09:22.544 DEBUG:teuthology.orchestra.run.smithi150:> true 2024-09-10T18:09:22.925 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-10T18:09:22.925 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-10T18:09:22.928 DEBUG:teuthology.orchestra.run.smithi150:> uname -r 2024-09-10T18:09:22.983 INFO:teuthology.orchestra.run.smithi150.stdout:5.14.0-505.el9.x86_64 2024-09-10T18:09:22.984 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T18:09:22.984 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T18:09:22.984 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T18:09:23.985 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-10T18:09:23.985 DEBUG:teuthology.orchestra.run.smithi150:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T18:09:24.111 INFO:teuthology.orchestra.run.smithi150.stdout:ttyS1 2024-09-10T18:09:24.132 DEBUG:teuthology.parallel:result is None 2024-09-10T18:09:24.132 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T18:09:24.141 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T18:09:24.141 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T18:09:24.144 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T18:09:24.190 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T18:09:24.201 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T18:09:24.228 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T18:09:24.228 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T18:09:24.230 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T18:09:24.299 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T18:09:24.310 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T18:09:24.311 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T18:09:24.313 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T18:09:24.339 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T18:09:24.350 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T18:09:24.408 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T18:09:24.429 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T18:09:24.434 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T18:09:24.445 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T18:09:24.445 DEBUG:teuthology.orchestra.run.smithi115:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T18:09:24.448 DEBUG:teuthology.orchestra.run.smithi150:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T18:09:24.522 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T18:09:24.533 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T18:09:24.533 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T18:09:24.535 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T18:09:24.577 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T18:09:24.628 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T18:09:24.696 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T18:09:24.807 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T18:09:24.898 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T18:09:24.898 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T18:09:24.959 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T18:09:24.996 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T18:09:25.186 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T18:09:25.226 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T18:09:25.273 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-10T18:09:25.273 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T18:09:25.343 DEBUG:teuthology.orchestra.run.smithi115:> sudo service rsyslog restart 2024-09-10T18:09:25.345 DEBUG:teuthology.orchestra.run.smithi150:> sudo service rsyslog restart 2024-09-10T18:09:25.427 INFO:teuthology.orchestra.run.smithi115.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T18:09:25.428 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T18:09:25.859 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T18:09:25.867 INFO:teuthology.task.internal:Starting timer... 2024-09-10T18:09:25.867 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T18:09:25.897 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T18:09:25.928 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-10T18:09:25.929 DEBUG:teuthology.orchestra.run.smithi115:> sudo service auditd rotate 2024-09-10T18:09:26.126 INFO:teuthology.orchestra.run.smithi115.stdout:Rotating logs: 2024-09-10T18:09:26.128 DEBUG:teuthology.orchestra.run.smithi150:> sudo service auditd rotate 2024-09-10T18:09:26.174 INFO:teuthology.orchestra.run.smithi150.stdout:Rotating logs: 2024-09-10T18:09:26.176 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T18:09:26.176 DEBUG:teuthology.orchestra.run.smithi115:> /usr/sbin/getenforce 2024-09-10T18:09:26.209 INFO:teuthology.orchestra.run.smithi115.stdout:Permissive 2024-09-10T18:09:26.209 DEBUG:teuthology.orchestra.run.smithi150:> /usr/sbin/getenforce 2024-09-10T18:09:26.237 INFO:teuthology.orchestra.run.smithi150.stdout:Permissive 2024-09-10T18:09:26.237 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi115.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi150.front.sepia.ceph.com': 'permissive'} 2024-09-10T18:09:26.237 DEBUG:teuthology.orchestra.run.smithi115:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T18:09:26.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:09:26.280 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T18:09:26.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:09:26.306 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T18:09:26.306 DEBUG:teuthology.orchestra.run.smithi115:> sudo /usr/sbin/setenforce permissive 2024-09-10T18:09:26.376 DEBUG:teuthology.orchestra.run.smithi150:> sudo /usr/sbin/setenforce permissive 2024-09-10T18:09:26.412 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T18:09:26.423 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T18:09:26.434 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T18:09:26.435 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi115.front.sepia.ceph.com,smithi150.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T18:13:06.576 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi115.front.sepia.ceph.com'), Remote(name='ubuntu@smithi150.front.sepia.ceph.com')] 2024-09-10T18:13:06.577 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-10T18:13:06.578 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:13:06.656 DEBUG:teuthology.orchestra.run.smithi115:> true 2024-09-10T18:13:06.732 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-10T18:13:06.732 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-10T18:13:06.733 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:13:06.808 DEBUG:teuthology.orchestra.run.smithi150:> true 2024-09-10T18:13:06.841 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-10T18:13:06.842 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T18:13:06.852 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T18:13:06.852 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T18:13:06.852 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T18:13:06.855 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T18:13:06.855 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T18:13:06.881 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T18:13:06.896 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T18:13:06.910 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T18:13:06.920 INFO:teuthology.orchestra.run.smithi115.stderr:sudo: ntpd: command not found 2024-09-10T18:13:06.925 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T18:13:06.931 INFO:teuthology.orchestra.run.smithi115.stdout:506 Cannot talk to daemon 2024-09-10T18:13:06.945 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T18:13:06.949 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: ntpd: command not found 2024-09-10T18:13:06.959 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T18:13:06.960 INFO:teuthology.orchestra.run.smithi150.stdout:506 Cannot talk to daemon 2024-09-10T18:13:06.975 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T18:13:06.989 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T18:13:07.021 INFO:teuthology.orchestra.run.smithi115.stderr:bash: line 1: ntpq: command not found 2024-09-10T18:13:07.024 INFO:teuthology.orchestra.run.smithi115.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T18:13:07.024 INFO:teuthology.orchestra.run.smithi115.stdout:=============================================================================== 2024-09-10T18:13:07.024 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:13:07.024 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:13:07.024 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:13:07.024 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:13:07.045 INFO:teuthology.orchestra.run.smithi150.stderr:bash: line 1: ntpq: command not found 2024-09-10T18:13:07.048 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T18:13:07.048 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-09-10T18:13:07.048 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:13:07.048 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:13:07.048 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:13:07.048 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:13:07.049 INFO:teuthology.run_tasks:Running task pexec... 2024-09-10T18:13:07.059 INFO:teuthology.task.pexec:Executing custom commands... 2024-09-10T18:13:07.059 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi115.front.sepia.ceph.com 2024-09-10T18:13:07.060 DEBUG:teuthology.orchestra.run.smithi115:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-10T18:13:07.060 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi150.front.sepia.ceph.com 2024-09-10T18:13:07.060 DEBUG:teuthology.orchestra.run.smithi150:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-10T18:13:07.417 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T18:13:07.435 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:13:07.435 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2024-09-10T18:13:07.435 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:13:07.435 INFO:teuthology.orchestra.run.smithi115.stdout:Removing: 2024-09-10T18:13:07.435 INFO:teuthology.orchestra.run.smithi115.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-10T18:13:07.435 INFO:teuthology.orchestra.run.smithi115.stdout:Removing unused dependencies: 2024-09-10T18:13:07.435 INFO:teuthology.orchestra.run.smithi115.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-10T18:13:07.436 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:13:07.436 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T18:13:07.436 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:13:07.436 INFO:teuthology.orchestra.run.smithi115.stdout:Remove 2 Packages 2024-09-10T18:13:07.436 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:13:07.436 INFO:teuthology.orchestra.run.smithi115.stdout:Freed space: 5.5 M 2024-09-10T18:13:07.436 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T18:13:07.439 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T18:13:07.440 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T18:13:07.440 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T18:13:07.447 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:13:07.447 INFO:teuthology.orchestra.run.smithi150.stdout: Package Architecture Version Repository Size 2024-09-10T18:13:07.447 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:13:07.447 INFO:teuthology.orchestra.run.smithi150.stdout:Removing: 2024-09-10T18:13:07.447 INFO:teuthology.orchestra.run.smithi150.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-10T18:13:07.447 INFO:teuthology.orchestra.run.smithi150.stdout:Removing unused dependencies: 2024-09-10T18:13:07.447 INFO:teuthology.orchestra.run.smithi150.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-10T18:13:07.447 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:13:07.448 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T18:13:07.448 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:13:07.448 INFO:teuthology.orchestra.run.smithi150.stdout:Remove 2 Packages 2024-09-10T18:13:07.448 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:13:07.448 INFO:teuthology.orchestra.run.smithi150.stdout:Freed space: 5.5 M 2024-09-10T18:13:07.448 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T18:13:07.452 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T18:13:07.452 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T18:13:07.498 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T18:13:07.498 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T18:13:07.508 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T18:13:07.509 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T18:13:07.642 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T18:13:07.652 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T18:13:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-10T18:13:07.746 INFO:teuthology.orchestra.run.smithi115.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-10T18:13:07.787 INFO:teuthology.orchestra.run.smithi150.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-10T18:13:07.812 INFO:teuthology.orchestra.run.smithi115.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-10T18:13:08.255 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-10T18:13:08.255 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-10T18:13:08.355 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-10T18:13:08.355 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-10T18:13:08.532 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-10T18:13:08.533 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:13:08.533 INFO:teuthology.orchestra.run.smithi115.stdout:Removed: 2024-09-10T18:13:08.533 INFO:teuthology.orchestra.run.smithi115.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-10T18:13:08.533 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:13:08.533 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T18:13:08.664 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-10T18:13:08.664 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:13:08.664 INFO:teuthology.orchestra.run.smithi150.stdout:Removed: 2024-09-10T18:13:08.665 INFO:teuthology.orchestra.run.smithi150.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-10T18:13:08.665 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:13:08.665 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T18:13:09.411 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:47 ago on Tue 10 Sep 2024 06:12:22 PM UTC. 2024-09-10T18:13:09.484 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:47 ago on Tue 10 Sep 2024 06:12:22 PM UTC. 2024-09-10T18:13:09.544 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T18:13:09.544 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:13:09.544 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-10T18:13:09.545 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:13:09.545 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T18:13:09.545 INFO:teuthology.orchestra.run.smithi115.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-10T18:13:09.545 INFO:teuthology.orchestra.run.smithi115.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-10T18:13:09.545 INFO:teuthology.orchestra.run.smithi115.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-10T18:13:09.545 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-10T18:13:09.545 INFO:teuthology.orchestra.run.smithi115.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-10T18:13:09.545 INFO:teuthology.orchestra.run.smithi115.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-10T18:13:09.545 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-10T18:13:09.545 INFO:teuthology.orchestra.run.smithi115.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-10T18:13:09.545 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:13:09.545 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T18:13:09.545 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:13:09.545 INFO:teuthology.orchestra.run.smithi115.stdout:Install 7 Packages 2024-09-10T18:13:09.546 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:13:09.561 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 5.2 M 2024-09-10T18:13:09.561 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 19 M 2024-09-10T18:13:09.561 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T18:13:09.632 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T18:13:09.632 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:13:09.633 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-09-10T18:13:09.633 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:13:09.633 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-10T18:13:09.633 INFO:teuthology.orchestra.run.smithi150.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-10T18:13:09.633 INFO:teuthology.orchestra.run.smithi150.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-10T18:13:09.633 INFO:teuthology.orchestra.run.smithi150.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-10T18:13:09.633 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-09-10T18:13:09.633 INFO:teuthology.orchestra.run.smithi150.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-10T18:13:09.633 INFO:teuthology.orchestra.run.smithi150.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-10T18:13:09.633 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-10T18:13:09.633 INFO:teuthology.orchestra.run.smithi150.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-10T18:13:09.633 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:13:09.633 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T18:13:09.634 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:13:09.634 INFO:teuthology.orchestra.run.smithi150.stdout:Install 7 Packages 2024-09-10T18:13:09.634 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:13:09.646 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 5.2 M 2024-09-10T18:13:09.646 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 19 M 2024-09-10T18:13:09.646 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-10T18:13:09.851 INFO:teuthology.orchestra.run.smithi115.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 387 kB/s | 44 kB 00:00 2024-09-10T18:13:09.876 INFO:teuthology.orchestra.run.smithi115.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 756 kB/s | 106 kB 00:00 2024-09-10T18:13:09.927 INFO:teuthology.orchestra.run.smithi115.stdout:(3/7): python3-configshell-1.1.30-1.el9.noarch. 952 kB/s | 72 kB 00:00 2024-09-10T18:13:09.952 INFO:teuthology.orchestra.run.smithi115.stdout:(4/7): python3-kmod-0.9-32.el9.x86_64.rpm 1.1 MB/s | 84 kB 00:00 2024-09-10T18:13:09.960 INFO:teuthology.orchestra.run.smithi150.stdout:(1/7): libnvme-1.9-3.el9.x86_64.rpm 632 kB/s | 106 kB 00:00 2024-09-10T18:13:09.986 INFO:teuthology.orchestra.run.smithi115.stdout:(5/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 3.5 MB/s | 906 kB 00:00 2024-09-10T18:13:09.994 INFO:teuthology.orchestra.run.smithi150.stdout:(2/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 4.4 MB/s | 906 kB 00:00 2024-09-10T18:13:10.011 INFO:teuthology.orchestra.run.smithi150.stdout:(3/7): nvmetcli-0.7-3.el9.noarch.rpm 204 kB/s | 44 kB 00:00 2024-09-10T18:13:10.045 INFO:teuthology.orchestra.run.smithi150.stdout:(4/7): python3-configshell-1.1.30-1.el9.noarch. 856 kB/s | 72 kB 00:00 2024-09-10T18:13:10.070 INFO:teuthology.orchestra.run.smithi150.stdout:(5/7): python3-kmod-0.9-32.el9.x86_64.rpm 1.1 MB/s | 84 kB 00:00 2024-09-10T18:13:10.102 INFO:teuthology.orchestra.run.smithi115.stdout:(6/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 4.7 MB/s | 837 kB 00:00 2024-09-10T18:13:10.145 INFO:teuthology.orchestra.run.smithi150.stdout:(6/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 6.1 MB/s | 837 kB 00:00 2024-09-10T18:13:11.995 INFO:teuthology.orchestra.run.smithi115.stdout:(7/7): runc-1.1.13-4.el9.x86_64.rpm 1.5 MB/s | 3.2 MB 00:02 2024-09-10T18:13:11.995 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:13:11.995 INFO:teuthology.orchestra.run.smithi115.stdout:Total 2.1 MB/s | 5.2 MB 00:02 2024-09-10T18:13:12.109 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T18:13:12.125 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T18:13:12.125 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T18:13:12.439 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T18:13:12.439 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T18:13:12.918 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T18:13:13.004 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-10T18:13:13.039 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-10T18:13:13.068 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-10T18:13:13.276 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-10T18:13:13.307 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-10T18:13:13.695 INFO:teuthology.orchestra.run.smithi150.stdout:(7/7): runc-1.1.13-4.el9.x86_64.rpm 888 kB/s | 3.2 MB 00:03 2024-09-10T18:13:13.696 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:13:13.696 INFO:teuthology.orchestra.run.smithi150.stdout:Total 1.3 MB/s | 5.2 MB 00:04 2024-09-10T18:13:13.749 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-10T18:13:13.777 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-10T18:13:13.815 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T18:13:13.831 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T18:13:13.831 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T18:13:13.894 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-10T18:13:13.955 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-10T18:13:14.117 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T18:13:14.117 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T18:13:14.583 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T18:13:14.664 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-10T18:13:14.699 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-10T18:13:14.728 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-10T18:13:14.859 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-10T18:13:14.859 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-10T18:13:14.859 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-10T18:13:14.859 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-10T18:13:14.860 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-10T18:13:14.860 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-10T18:13:14.860 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-10T18:13:14.946 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-10T18:13:14.975 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-10T18:13:15.254 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-10T18:13:15.255 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:13:15.255 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T18:13:15.255 INFO:teuthology.orchestra.run.smithi115.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-10T18:13:15.255 INFO:teuthology.orchestra.run.smithi115.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-10T18:13:15.255 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-10T18:13:15.255 INFO:teuthology.orchestra.run.smithi115.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-10T18:13:15.256 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:13:15.256 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T18:13:15.416 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-10T18:13:15.444 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-10T18:13:15.566 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-10T18:13:15.634 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-10T18:13:15.639 DEBUG:teuthology.parallel:result is None 2024-09-10T18:13:16.604 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-10T18:13:16.604 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-10T18:13:16.604 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-10T18:13:16.604 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-10T18:13:16.605 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-10T18:13:16.605 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-10T18:13:16.605 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-10T18:13:17.000 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-10T18:13:17.000 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:13:17.000 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-10T18:13:17.000 INFO:teuthology.orchestra.run.smithi150.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-10T18:13:17.000 INFO:teuthology.orchestra.run.smithi150.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-10T18:13:17.001 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-10T18:13:17.001 INFO:teuthology.orchestra.run.smithi150.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-10T18:13:17.001 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:13:17.001 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T18:13:17.407 DEBUG:teuthology.parallel:result is None 2024-09-10T18:13:17.407 INFO:teuthology.run_tasks:Running task install... 2024-09-10T18:13:17.414 DEBUG:teuthology.task.install:project ceph 2024-09-10T18:13:17.415 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588'}, 'extra_packages': ['cephadm']} 2024-09-10T18:13:17.415 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588'} 2024-09-10T18:13:17.415 INFO:teuthology.task.install:Using flavor: default 2024-09-10T18:13:17.420 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T18:13:17.420 INFO:teuthology.task.install:extra packages: [] 2024-09-10T18:13:17.421 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588', 'tag': None, 'wait_for_package': False} 2024-09-10T18:13:17.421 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:13:17.423 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588', 'tag': None, 'wait_for_package': False} 2024-09-10T18:13:17.423 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:13:17.571 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/ 2024-09-10T18:13:17.571 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1606.g5be57395 2024-09-10T18:13:17.581 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/ 2024-09-10T18:13:17.581 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1606.g5be57395 2024-09-10T18:13:17.681 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T18:13:17.682 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T18:13:17.682 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T18:13:17.693 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T18:13:17.693 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-10T18:13:17.693 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T18:13:17.713 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-10T18:13:17.714 DEBUG:teuthology.orchestra.run.smithi115:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5be57395ea235535910512ae73ba8e7ff25fe588/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T18:13:17.719 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-10T18:13:17.719 DEBUG:teuthology.orchestra.run.smithi150:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5be57395ea235535910512ae73ba8e7ff25fe588/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T18:13:17.787 DEBUG:teuthology.orchestra.run.smithi115:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T18:13:17.788 DEBUG:teuthology.orchestra.run.smithi150:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T18:13:17.862 DEBUG:teuthology.orchestra.run.smithi115:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T18:13:17.864 DEBUG:teuthology.orchestra.run.smithi150:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T18:13:17.929 INFO:teuthology.orchestra.run.smithi150.stdout:check_obsoletes = 1 2024-09-10T18:13:17.929 INFO:teuthology.orchestra.run.smithi115.stdout:check_obsoletes = 1 2024-09-10T18:13:17.930 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum clean all 2024-09-10T18:13:17.931 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum clean all 2024-09-10T18:13:18.155 INFO:teuthology.orchestra.run.smithi115.stdout:45 files removed 2024-09-10T18:13:18.173 INFO:teuthology.orchestra.run.smithi150.stdout:45 files removed 2024-09-10T18:13:18.182 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-radosgw 2024-09-10T18:13:18.204 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-radosgw 2024-09-10T18:13:18.735 INFO:teuthology.orchestra.run.smithi115.stdout:ceph packages for x86_64 290 kB/s | 77 kB 00:00 2024-09-10T18:13:18.746 INFO:teuthology.orchestra.run.smithi150.stdout:ceph packages for x86_64 303 kB/s | 77 kB 00:00 2024-09-10T18:13:19.013 INFO:teuthology.orchestra.run.smithi115.stdout:ceph noarch packages 47 kB/s | 11 kB 00:00 2024-09-10T18:13:19.030 INFO:teuthology.orchestra.run.smithi150.stdout:ceph noarch packages 45 kB/s | 11 kB 00:00 2024-09-10T18:13:19.270 INFO:teuthology.orchestra.run.smithi115.stdout:ceph source packages 8.0 kB/s | 1.8 kB 00:00 2024-09-10T18:13:19.297 INFO:teuthology.orchestra.run.smithi150.stdout:ceph source packages 7.8 kB/s | 1.8 kB 00:00 2024-09-10T18:13:19.755 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - BaseOS 19 MB/s | 8.2 MB 00:00 2024-09-10T18:13:19.829 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.2 MB 00:00 2024-09-10T18:13:21.904 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - AppStream 22 MB/s | 20 MB 00:00 2024-09-10T18:13:26.015 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - AppStream 4.0 MB/s | 20 MB 00:05 2024-09-10T18:13:30.532 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - CRB 9.5 MB/s | 6.5 MB 00:00 2024-09-10T18:13:31.908 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - Extras packages 130 kB/s | 19 kB 00:00 2024-09-10T18:13:32.537 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - CRB 978 kB/s | 6.5 MB 00:06 2024-09-10T18:13:33.037 INFO:teuthology.orchestra.run.smithi150.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-10T18:13:34.121 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - Extras packages 83 kB/s | 19 kB 00:00 2024-09-10T18:13:34.858 INFO:teuthology.orchestra.run.smithi115.stdout:Extra Packages for Enterprise Linux 37 MB/s | 23 MB 00:00 2024-09-10T18:13:38.493 INFO:teuthology.orchestra.run.smithi150.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-09-10T18:13:40.281 INFO:teuthology.orchestra.run.smithi115.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-09-10T18:13:40.864 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T18:13:40.866 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:13:40.866 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-09-10T18:13:40.866 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:13:40.866 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-10T18:13:40.866 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw x86_64 2:17.2.7-1606.g5be57395.el9 ceph 14 M 2024-09-10T18:13:40.866 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrading: 2024-09-10T18:13:40.867 INFO:teuthology.orchestra.run.smithi150.stdout: librados2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 3.1 M 2024-09-10T18:13:40.867 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.9 M 2024-09-10T18:13:40.867 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-09-10T18:13:40.867 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base x86_64 2:17.2.7-1606.g5be57395.el9 ceph 5.0 M 2024-09-10T18:13:40.867 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common x86_64 2:17.2.7-1606.g5be57395.el9 ceph 22 M 2024-09-10T18:13:40.867 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux x86_64 2:17.2.7-1606.g5be57395.el9 ceph 25 k 2024-09-10T18:13:40.867 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T18:13:40.867 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 637 k 2024-09-10T18:13:40.867 INFO:teuthology.orchestra.run.smithi150.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T18:13:40.867 INFO:teuthology.orchestra.run.smithi150.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T18:13:40.867 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T18:13:40.867 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 458 k 2024-09-10T18:13:40.867 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T18:13:40.868 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 4.9 M 2024-09-10T18:13:40.868 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T18:13:40.868 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T18:13:40.868 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 45 k 2024-09-10T18:13:40.868 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common x86_64 2:17.2.7-1606.g5be57395.el9 ceph 120 k 2024-09-10T18:13:40.868 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs x86_64 2:17.2.7-1606.g5be57395.el9 ceph 156 k 2024-09-10T18:13:40.868 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T18:13:40.868 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados x86_64 2:17.2.7-1606.g5be57395.el9 ceph 320 k 2024-09-10T18:13:40.868 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 295 k 2024-09-10T18:13:40.868 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw x86_64 2:17.2.7-1606.g5be57395.el9 ceph 99 k 2024-09-10T18:13:40.868 INFO:teuthology.orchestra.run.smithi150.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T18:13:40.868 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:13:40.868 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T18:13:40.869 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:13:40.869 INFO:teuthology.orchestra.run.smithi150.stdout:Install 22 Packages 2024-09-10T18:13:40.869 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrade 2 Packages 2024-09-10T18:13:40.869 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:13:40.869 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 58 M 2024-09-10T18:13:40.869 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-10T18:13:41.856 INFO:teuthology.orchestra.run.smithi150.stdout:(1/24): ceph-base-17.2.7-1606.g5be57395.el9.x86 11 MB/s | 5.0 MB 00:00 2024-09-10T18:13:41.903 INFO:teuthology.orchestra.run.smithi150.stdout:(2/24): ceph-selinux-17.2.7-1606.g5be57395.el9. 504 kB/s | 25 kB 00:00 2024-09-10T18:13:41.986 INFO:teuthology.orchestra.run.smithi150.stdout:(3/24): libcephfs2-17.2.7-1606.g5be57395.el9.x8 7.5 MB/s | 637 kB 00:00 2024-09-10T18:13:42.078 INFO:teuthology.orchestra.run.smithi150.stdout:(4/24): libradosstriper1-17.2.7-1606.g5be57395. 4.9 MB/s | 458 kB 00:00 2024-09-10T18:13:42.387 INFO:teuthology.orchestra.run.smithi150.stdout:(5/24): ceph-radosgw-17.2.7-1606.g5be57395.el9. 14 MB/s | 14 MB 00:01 2024-09-10T18:13:42.462 INFO:teuthology.orchestra.run.smithi150.stdout:(6/24): librgw2-17.2.7-1606.g5be57395.el9.x86_6 13 MB/s | 4.9 MB 00:00 2024-09-10T18:13:42.479 INFO:teuthology.orchestra.run.smithi150.stdout:(7/24): python3-ceph-argparse-17.2.7-1606.g5be5 492 kB/s | 45 kB 00:00 2024-09-10T18:13:42.504 INFO:teuthology.orchestra.run.smithi150.stdout:(8/24): python3-ceph-common-17.2.7-1606.g5be573 2.8 MB/s | 120 kB 00:00 2024-09-10T18:13:42.546 INFO:teuthology.orchestra.run.smithi150.stdout:(9/24): python3-cephfs-17.2.7-1606.g5be57395.el 2.3 MB/s | 156 kB 00:00 2024-09-10T18:13:42.572 INFO:teuthology.orchestra.run.smithi150.stdout:(10/24): python3-rados-17.2.7-1606.g5be57395.el 4.7 MB/s | 320 kB 00:00 2024-09-10T18:13:42.647 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T18:13:42.649 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:13:42.649 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-10T18:13:42.649 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:13:42.649 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T18:13:42.649 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-radosgw x86_64 2:17.2.7-1606.g5be57395.el9 ceph 14 M 2024-09-10T18:13:42.649 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrading: 2024-09-10T18:13:42.649 INFO:teuthology.orchestra.run.smithi115.stdout: librados2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 3.1 M 2024-09-10T18:13:42.650 INFO:teuthology.orchestra.run.smithi115.stdout: librbd1 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.9 M 2024-09-10T18:13:42.650 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-10T18:13:42.650 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base x86_64 2:17.2.7-1606.g5be57395.el9 ceph 5.0 M 2024-09-10T18:13:42.650 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-common x86_64 2:17.2.7-1606.g5be57395.el9 ceph 22 M 2024-09-10T18:13:42.650 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-selinux x86_64 2:17.2.7-1606.g5be57395.el9 ceph 25 k 2024-09-10T18:13:42.650 INFO:teuthology.orchestra.run.smithi115.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T18:13:42.650 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 637 k 2024-09-10T18:13:42.650 INFO:teuthology.orchestra.run.smithi115.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T18:13:42.650 INFO:teuthology.orchestra.run.smithi115.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T18:13:42.650 INFO:teuthology.orchestra.run.smithi115.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T18:13:42.650 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 458 k 2024-09-10T18:13:42.650 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T18:13:42.651 INFO:teuthology.orchestra.run.smithi115.stdout: librgw2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 4.9 M 2024-09-10T18:13:42.651 INFO:teuthology.orchestra.run.smithi115.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T18:13:42.651 INFO:teuthology.orchestra.run.smithi115.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T18:13:42.651 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 45 k 2024-09-10T18:13:42.651 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common x86_64 2:17.2.7-1606.g5be57395.el9 ceph 120 k 2024-09-10T18:13:42.651 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cephfs x86_64 2:17.2.7-1606.g5be57395.el9 ceph 156 k 2024-09-10T18:13:42.651 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T18:13:42.651 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rados x86_64 2:17.2.7-1606.g5be57395.el9 ceph 320 k 2024-09-10T18:13:42.651 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rbd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 295 k 2024-09-10T18:13:42.651 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rgw x86_64 2:17.2.7-1606.g5be57395.el9 ceph 99 k 2024-09-10T18:13:42.651 INFO:teuthology.orchestra.run.smithi115.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T18:13:42.651 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:13:42.651 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T18:13:42.652 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:13:42.652 INFO:teuthology.orchestra.run.smithi115.stdout:Install 22 Packages 2024-09-10T18:13:42.652 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrade 2 Packages 2024-09-10T18:13:42.652 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:13:42.652 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 58 M 2024-09-10T18:13:42.652 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T18:13:42.797 INFO:teuthology.orchestra.run.smithi150.stdout:(11/24): ceph-common-17.2.7-1606.g5be57395.el9. 16 MB/s | 22 MB 00:01 2024-09-10T18:13:42.822 INFO:teuthology.orchestra.run.smithi150.stdout:(12/24): python3-rgw-17.2.7-1606.g5be57395.el9. 397 kB/s | 99 kB 00:00 2024-09-10T18:13:42.847 INFO:teuthology.orchestra.run.smithi150.stdout:(13/24): python3-rbd-17.2.7-1606.g5be57395.el9. 979 kB/s | 295 kB 00:00 2024-09-10T18:13:42.964 INFO:teuthology.orchestra.run.smithi150.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 319 kB/s | 45 kB 00:00 2024-09-10T18:13:43.015 INFO:teuthology.orchestra.run.smithi150.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 737 kB/s | 160 kB 00:00 2024-09-10T18:13:43.065 INFO:teuthology.orchestra.run.smithi150.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 850 kB/s | 42 kB 00:00 2024-09-10T18:13:43.115 INFO:teuthology.orchestra.run.smithi150.stdout:(17/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 6.0 MB/s | 309 kB 00:00 2024-09-10T18:13:43.140 INFO:teuthology.orchestra.run.smithi150.stdout:(18/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.6 MB/s | 292 kB 00:00 2024-09-10T18:13:43.174 INFO:teuthology.orchestra.run.smithi150.stdout:(19/24): librdkafka-1.6.1-102.el9.x86_64.rpm 2.0 MB/s | 662 kB 00:00 2024-09-10T18:13:43.208 INFO:teuthology.orchestra.run.smithi150.stdout:(20/24): liboath-2.6.7-2.el9.x86_64.rpm 528 kB/s | 49 kB 00:00 2024-09-10T18:13:43.241 INFO:teuthology.orchestra.run.smithi150.stdout:(21/24): libunwind-1.6.2-1.el9.x86_64.rpm 669 kB/s | 67 kB 00:00 2024-09-10T18:13:43.291 INFO:teuthology.orchestra.run.smithi150.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-10T18:13:43.323 INFO:teuthology.orchestra.run.smithi115.stdout:(1/24): ceph-base-17.2.7-1606.g5be57395.el9.x86 11 MB/s | 5.0 MB 00:00 2024-09-10T18:13:43.423 INFO:teuthology.orchestra.run.smithi115.stdout:(2/24): ceph-selinux-17.2.7-1606.g5be57395.el9. 252 kB/s | 25 kB 00:00 2024-09-10T18:13:43.475 INFO:teuthology.orchestra.run.smithi150.stdout:(23/24): librados2-17.2.7-1606.g5be57395.el9.x8 12 MB/s | 3.1 MB 00:00 2024-09-10T18:13:43.524 INFO:teuthology.orchestra.run.smithi115.stdout:(3/24): libcephfs2-17.2.7-1606.g5be57395.el9.x8 6.2 MB/s | 637 kB 00:00 2024-09-10T18:13:43.592 INFO:teuthology.orchestra.run.smithi150.stdout:(24/24): librbd1-17.2.7-1606.g5be57395.el9.x86_ 8.4 MB/s | 2.9 MB 00:00 2024-09-10T18:13:43.597 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:13:43.597 INFO:teuthology.orchestra.run.smithi150.stdout:Total 21 MB/s | 58 MB 00:02 2024-09-10T18:13:43.616 INFO:teuthology.orchestra.run.smithi115.stdout:(4/24): libradosstriper1-17.2.7-1606.g5be57395. 4.9 MB/s | 458 kB 00:00 2024-09-10T18:13:43.658 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T18:13:43.692 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T18:13:43.692 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T18:13:43.999 INFO:teuthology.orchestra.run.smithi115.stdout:(5/24): librgw2-17.2.7-1606.g5be57395.el9.x86_6 13 MB/s | 4.9 MB 00:00 2024-09-10T18:13:44.173 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T18:13:44.173 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T18:13:44.216 INFO:teuthology.orchestra.run.smithi115.stdout:(6/24): ceph-common-17.2.7-1606.g5be57395.el9.x 17 MB/s | 22 MB 00:01 2024-09-10T18:13:44.242 INFO:teuthology.orchestra.run.smithi115.stdout:(7/24): python3-ceph-argparse-17.2.7-1606.g5be5 186 kB/s | 45 kB 00:00 2024-09-10T18:13:44.267 INFO:teuthology.orchestra.run.smithi115.stdout:(8/24): python3-ceph-common-17.2.7-1606.g5be573 2.4 MB/s | 120 kB 00:00 2024-09-10T18:13:44.292 INFO:teuthology.orchestra.run.smithi115.stdout:(9/24): python3-cephfs-17.2.7-1606.g5be57395.el 3.0 MB/s | 156 kB 00:00 2024-09-10T18:13:44.317 INFO:teuthology.orchestra.run.smithi115.stdout:(10/24): python3-rados-17.2.7-1606.g5be57395.el 6.2 MB/s | 320 kB 00:00 2024-09-10T18:13:44.343 INFO:teuthology.orchestra.run.smithi115.stdout:(11/24): python3-rbd-17.2.7-1606.g5be57395.el9. 5.7 MB/s | 295 kB 00:00 2024-09-10T18:13:44.368 INFO:teuthology.orchestra.run.smithi115.stdout:(12/24): python3-rgw-17.2.7-1606.g5be57395.el9. 1.9 MB/s | 99 kB 00:00 2024-09-10T18:13:44.510 INFO:teuthology.orchestra.run.smithi115.stdout:(13/24): ceph-radosgw-17.2.7-1606.g5be57395.el9 8.7 MB/s | 14 MB 00:01 2024-09-10T18:13:44.785 INFO:teuthology.orchestra.run.smithi115.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 108 kB/s | 45 kB 00:00 2024-09-10T18:13:44.969 INFO:teuthology.orchestra.run.smithi115.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 256 kB/s | 160 kB 00:00 2024-09-10T18:13:45.077 INFO:teuthology.orchestra.run.smithi115.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 390 kB/s | 42 kB 00:00 2024-09-10T18:13:45.127 INFO:teuthology.orchestra.run.smithi115.stdout:(17/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 6.1 MB/s | 309 kB 00:00 2024-09-10T18:13:45.144 INFO:teuthology.orchestra.run.smithi115.stdout:(18/24): liboath-2.6.7-2.el9.x86_64.rpm 2.8 MB/s | 49 kB 00:00 2024-09-10T18:13:45.170 INFO:teuthology.orchestra.run.smithi115.stdout:(19/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 761 kB/s | 292 kB 00:00 2024-09-10T18:13:45.197 INFO:teuthology.orchestra.run.smithi115.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-10T18:13:45.270 INFO:teuthology.orchestra.run.smithi115.stdout:(21/24): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-10T18:13:45.337 INFO:teuthology.orchestra.run.smithi115.stdout:(22/24): librdkafka-1.6.1-102.el9.x86_64.rpm 801 kB/s | 662 kB 00:00 2024-09-10T18:13:45.348 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T18:13:45.596 INFO:teuthology.orchestra.run.smithi115.stdout:(23/24): librados2-17.2.7-1606.g5be57395.el9.x8 7.8 MB/s | 3.1 MB 00:00 2024-09-10T18:13:45.698 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-10T18:13:45.704 INFO:teuthology.orchestra.run.smithi115.stdout:(24/24): librbd1-17.2.7-1606.g5be57395.el9.x86_ 6.8 MB/s | 2.9 MB 00:00 2024-09-10T18:13:45.709 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:13:45.709 INFO:teuthology.orchestra.run.smithi115.stdout:Total 19 MB/s | 58 MB 00:03 2024-09-10T18:13:45.774 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T18:13:45.809 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T18:13:45.809 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T18:13:45.811 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2/26 2024-09-10T18:13:45.914 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2/26 2024-09-10T18:13:45.998 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 3/26 2024-09-10T18:13:46.029 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 4/26 2024-09-10T18:13:46.116 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 4/26 2024-09-10T18:13:46.163 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-10T18:13:46.192 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-10T18:13:46.291 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T18:13:46.291 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T18:13:46.639 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-10T18:13:46.665 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 8/26 2024-09-10T18:13:46.728 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 8/26 2024-09-10T18:13:46.756 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-10T18:13:46.807 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9. 10/26 2024-09-10T18:13:46.845 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 11/26 2024-09-10T18:13:46.906 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 12/26 2024-09-10T18:13:46.926 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_6 13/26 2024-09-10T18:13:46.991 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_6 13/26 2024-09-10T18:13:47.032 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-10T18:13:47.093 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-09-10T18:13:47.125 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-10T18:13:47.423 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-10T18:13:47.471 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 18/26 2024-09-10T18:13:47.492 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T18:13:47.561 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 18/26 2024-09-10T18:13:47.635 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 19/26 2024-09-10T18:13:47.666 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x8 20/26 2024-09-10T18:13:47.879 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-10T18:13:47.923 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2/26 2024-09-10T18:13:48.000 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2/26 2024-09-10T18:13:48.079 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 3/26 2024-09-10T18:13:48.131 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 4/26 2024-09-10T18:13:48.190 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 4/26 2024-09-10T18:13:48.232 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-10T18:13:48.268 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-10T18:13:48.714 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-10T18:13:48.752 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 8/26 2024-09-10T18:13:48.815 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 8/26 2024-09-10T18:13:48.852 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-10T18:13:48.889 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9. 10/26 2024-09-10T18:13:48.923 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 11/26 2024-09-10T18:13:48.985 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 12/26 2024-09-10T18:13:49.013 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_6 13/26 2024-09-10T18:13:49.079 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_6 13/26 2024-09-10T18:13:49.125 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-10T18:13:49.156 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-09-10T18:13:49.188 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-10T18:13:49.475 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-10T18:13:49.518 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 18/26 2024-09-10T18:13:49.608 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 18/26 2024-09-10T18:13:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 19/26 2024-09-10T18:13:49.711 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x8 20/26 2024-09-10T18:13:50.131 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 21/26 2024-09-10T18:13:50.171 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 21/26 2024-09-10T18:13:50.691 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 21/26 2024-09-10T18:13:50.749 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 22/26 2024-09-10T18:13:50.829 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 22/26 2024-09-10T18:13:50.829 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T18:13:50.829 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T18:13:50.829 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:13:50.858 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 23/26 2024-09-10T18:13:52.163 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 21/26 2024-09-10T18:13:52.202 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 21/26 2024-09-10T18:13:52.748 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 21/26 2024-09-10T18:13:52.789 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 22/26 2024-09-10T18:13:52.867 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 22/26 2024-09-10T18:13:52.867 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T18:13:52.867 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T18:13:52.867 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:13:52.898 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 23/26 2024-09-10T18:14:00.935 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 23/26 2024-09-10T18:14:00.935 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /sys 2024-09-10T18:14:00.935 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /proc 2024-09-10T18:14:00.935 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /mnt 2024-09-10T18:14:00.935 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /var/tmp 2024-09-10T18:14:00.935 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /home 2024-09-10T18:14:00.935 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /root 2024-09-10T18:14:00.935 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /tmp 2024-09-10T18:14:00.935 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:02.242 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 24/26 2024-09-10T18:14:02.320 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 24/26 2024-09-10T18:14:02.320 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:14:02.321 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T18:14:02.321 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T18:14:02.321 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T18:14:02.321 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:02.322 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-10T18:14:02.428 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-10T18:14:02.428 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-10T18:14:02.755 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 23/26 2024-09-10T18:14:02.756 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /sys 2024-09-10T18:14:02.756 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /proc 2024-09-10T18:14:02.756 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /mnt 2024-09-10T18:14:02.756 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /var/tmp 2024-09-10T18:14:02.756 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /home 2024-09-10T18:14:02.756 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /root 2024-09-10T18:14:02.756 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /tmp 2024-09-10T18:14:02.756 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:03.634 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-10T18:14:03.634 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 1/26 2024-09-10T18:14:03.634 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2/26 2024-09-10T18:14:03.634 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 3/26 2024-09-10T18:14:03.635 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 4/26 2024-09-10T18:14:03.635 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 5/26 2024-09-10T18:14:03.635 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_6 6/26 2024-09-10T18:14:03.635 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 7/26 2024-09-10T18:14:03.635 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9. 8/26 2024-09-10T18:14:03.635 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x8 9/26 2024-09-10T18:14:03.635 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 10/26 2024-09-10T18:14:03.635 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 11/26 2024-09-10T18:14:03.635 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 12/26 2024-09-10T18:14:03.635 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 13/26 2024-09-10T18:14:03.635 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-10T18:14:03.635 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-10T18:14:03.635 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-10T18:14:03.636 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-10T18:14:03.636 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-10T18:14:03.636 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-09-10T18:14:03.636 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-10T18:14:03.636 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-10T18:14:03.636 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-10T18:14:03.636 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:17.2.7-1606.g5be57395.el9.x86_64 23/26 2024-09-10T18:14:03.636 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-10T18:14:03.636 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 25/26 2024-09-10T18:14:04.046 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 24/26 2024-09-10T18:14:04.148 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 24/26 2024-09-10T18:14:04.148 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:14:04.148 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T18:14:04.148 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T18:14:04.148 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T18:14:04.148 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:04.149 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-10T18:14:04.241 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-10T18:14:04.241 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-10T18:14:04.565 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-10T18:14:04.565 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:04.565 INFO:teuthology.orchestra.run.smithi150.stdout:Upgraded: 2024-09-10T18:14:04.565 INFO:teuthology.orchestra.run.smithi150.stdout: librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:04.566 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:04.566 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-10T18:14:04.566 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:04.566 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:04.566 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:04.566 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:04.566 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T18:14:04.566 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:04.566 INFO:teuthology.orchestra.run.smithi150.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T18:14:04.566 INFO:teuthology.orchestra.run.smithi150.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T18:14:04.566 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T18:14:04.566 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:04.566 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T18:14:04.567 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:04.567 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T18:14:04.567 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T18:14:04.567 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:04.567 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:04.567 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:04.567 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T18:14:04.567 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:04.567 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:04.567 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:04.567 INFO:teuthology.orchestra.run.smithi150.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T18:14:04.567 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:04.567 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T18:14:04.947 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-test 2024-09-10T18:14:05.519 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:27 ago on Tue 10 Sep 2024 06:13:38 PM UTC. 2024-09-10T18:14:05.546 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-10T18:14:05.546 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 1/26 2024-09-10T18:14:05.546 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2/26 2024-09-10T18:14:05.546 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 3/26 2024-09-10T18:14:05.547 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 4/26 2024-09-10T18:14:05.547 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 5/26 2024-09-10T18:14:05.547 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_6 6/26 2024-09-10T18:14:05.547 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 7/26 2024-09-10T18:14:05.547 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9. 8/26 2024-09-10T18:14:05.547 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x8 9/26 2024-09-10T18:14:05.547 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 10/26 2024-09-10T18:14:05.547 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 11/26 2024-09-10T18:14:05.547 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 12/26 2024-09-10T18:14:05.547 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 13/26 2024-09-10T18:14:05.547 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-10T18:14:05.547 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-10T18:14:05.547 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-10T18:14:05.548 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-10T18:14:05.548 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-10T18:14:05.548 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-09-10T18:14:05.548 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-10T18:14:05.548 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-10T18:14:05.548 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-10T18:14:05.548 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados2-2:17.2.7-1606.g5be57395.el9.x86_64 23/26 2024-09-10T18:14:05.548 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-10T18:14:05.549 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 25/26 2024-09-10T18:14:05.651 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T18:14:05.652 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:14:05.652 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-09-10T18:14:05.652 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:14:05.652 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-10T18:14:05.652 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test x86_64 2:17.2.7-1606.g5be57395.el9 ceph 48 M 2024-09-10T18:14:05.652 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-09-10T18:14:05.653 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite x86_64 2:17.2.7-1606.g5be57395.el9 ceph 163 k 2024-09-10T18:14:05.653 INFO:teuthology.orchestra.run.smithi150.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T18:14:05.653 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T18:14:05.653 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:05.653 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T18:14:05.653 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:14:05.653 INFO:teuthology.orchestra.run.smithi150.stdout:Install 4 Packages 2024-09-10T18:14:05.653 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:05.654 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 48 M 2024-09-10T18:14:05.654 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 193 M 2024-09-10T18:14:05.654 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-10T18:14:05.924 INFO:teuthology.orchestra.run.smithi150.stdout:(1/4): libcephsqlite-17.2.7-1606.g5be57395.el9. 750 kB/s | 163 kB 00:00 2024-09-10T18:14:06.299 INFO:teuthology.orchestra.run.smithi150.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 170 kB/s | 64 kB 00:00 2024-09-10T18:14:06.341 INFO:teuthology.orchestra.run.smithi150.stdout:(3/4): socat-1.7.4.1-6.el9.x86_64.rpm 480 kB/s | 304 kB 00:00 2024-09-10T18:14:06.397 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-10T18:14:06.397 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:06.397 INFO:teuthology.orchestra.run.smithi115.stdout:Upgraded: 2024-09-10T18:14:06.397 INFO:teuthology.orchestra.run.smithi115.stdout: librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:06.397 INFO:teuthology.orchestra.run.smithi115.stdout: librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:06.397 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T18:14:06.397 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:06.397 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:06.397 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:06.397 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:06.398 INFO:teuthology.orchestra.run.smithi115.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T18:14:06.398 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:06.398 INFO:teuthology.orchestra.run.smithi115.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T18:14:06.398 INFO:teuthology.orchestra.run.smithi115.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T18:14:06.398 INFO:teuthology.orchestra.run.smithi115.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T18:14:06.398 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:06.398 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T18:14:06.398 INFO:teuthology.orchestra.run.smithi115.stdout: librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:06.398 INFO:teuthology.orchestra.run.smithi115.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T18:14:06.398 INFO:teuthology.orchestra.run.smithi115.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T18:14:06.399 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:06.399 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:06.399 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:06.399 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T18:14:06.399 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:06.399 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:06.399 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:06.399 INFO:teuthology.orchestra.run.smithi115.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T18:14:06.399 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:06.399 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T18:14:06.785 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-test 2024-09-10T18:14:07.175 INFO:teuthology.orchestra.run.smithi150.stdout:(4/4): ceph-test-17.2.7-1606.g5be57395.el9.x86_ 33 MB/s | 48 MB 00:01 2024-09-10T18:14:07.177 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:14:07.177 INFO:teuthology.orchestra.run.smithi150.stdout:Total 32 MB/s | 48 MB 00:01 2024-09-10T18:14:07.204 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T18:14:07.222 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T18:14:07.222 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T18:14:07.350 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:27 ago on Tue 10 Sep 2024 06:13:40 PM UTC. 2024-09-10T18:14:07.483 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T18:14:07.483 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:14:07.483 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-10T18:14:07.484 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:14:07.484 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T18:14:07.484 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-test x86_64 2:17.2.7-1606.g5be57395.el9 ceph 48 M 2024-09-10T18:14:07.484 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-10T18:14:07.484 INFO:teuthology.orchestra.run.smithi115.stdout: libcephsqlite x86_64 2:17.2.7-1606.g5be57395.el9 ceph 163 k 2024-09-10T18:14:07.484 INFO:teuthology.orchestra.run.smithi115.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T18:14:07.484 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T18:14:07.485 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:07.485 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T18:14:07.485 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:14:07.485 INFO:teuthology.orchestra.run.smithi115.stdout:Install 4 Packages 2024-09-10T18:14:07.485 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:07.486 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 48 M 2024-09-10T18:14:07.486 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 193 M 2024-09-10T18:14:07.486 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T18:14:07.588 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T18:14:07.588 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T18:14:07.805 INFO:teuthology.orchestra.run.smithi115.stdout:(1/4): libcephsqlite-17.2.7-1606.g5be57395.el9. 753 kB/s | 163 kB 00:00 2024-09-10T18:14:07.838 INFO:teuthology.orchestra.run.smithi115.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.2 MB/s | 304 kB 00:00 2024-09-10T18:14:07.955 INFO:teuthology.orchestra.run.smithi115.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 423 kB/s | 64 kB 00:00 2024-09-10T18:14:08.109 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T18:14:08.203 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-10T18:14:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-10T18:14:08.268 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 3/4 2024-09-10T18:14:09.097 INFO:teuthology.orchestra.run.smithi115.stdout:(4/4): ceph-test-17.2.7-1606.g5be57395.el9.x86_ 32 MB/s | 48 MB 00:01 2024-09-10T18:14:09.099 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:14:09.099 INFO:teuthology.orchestra.run.smithi115.stdout:Total 30 MB/s | 48 MB 00:01 2024-09-10T18:14:09.127 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T18:14:09.146 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T18:14:09.147 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T18:14:09.512 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T18:14:09.512 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T18:14:10.035 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T18:14:10.133 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-10T18:14:10.179 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-10T18:14:10.207 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 3/4 2024-09-10T18:14:12.666 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 3/4 2024-09-10T18:14:12.742 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 4/4 2024-09-10T18:14:14.619 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 3/4 2024-09-10T18:14:14.707 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 4/4 2024-09-10T18:14:15.050 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 4/4 2024-09-10T18:14:15.050 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 1/4 2024-09-10T18:14:15.050 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 2/4 2024-09-10T18:14:15.050 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-10T18:14:15.371 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-10T18:14:15.371 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:15.371 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-10T18:14:15.371 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:15.371 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:15.371 INFO:teuthology.orchestra.run.smithi150.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T18:14:15.372 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T18:14:15.372 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:15.372 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T18:14:15.684 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph 2024-09-10T18:14:16.257 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:38 ago on Tue 10 Sep 2024 06:13:38 PM UTC. 2024-09-10T18:14:16.400 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T18:14:16.402 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:14:16.402 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-09-10T18:14:16.403 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:14:16.403 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-10T18:14:16.403 INFO:teuthology.orchestra.run.smithi150.stdout: ceph x86_64 2:17.2.7-1606.g5be57395.el9 ceph 6.5 k 2024-09-10T18:14:16.403 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-09-10T18:14:16.403 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.1 M 2024-09-10T18:14:16.403 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr x86_64 2:17.2.7-1606.g5be57395.el9 ceph 1.4 M 2024-09-10T18:14:16.403 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 240 k 2024-09-10T18:14:16.403 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon x86_64 2:17.2.7-1606.g5be57395.el9 ceph 3.9 M 2024-09-10T18:14:16.403 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 15 M 2024-09-10T18:14:16.403 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T18:14:16.403 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T18:14:16.403 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T18:14:16.404 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T18:14:16.404 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T18:14:16.404 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T18:14:16.404 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T18:14:16.404 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-collections 2024-09-10T18:14:16.404 INFO:teuthology.orchestra.run.smithi150.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-10T18:14:16.404 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T18:14:16.404 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T18:14:16.404 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T18:14:16.404 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T18:14:16.404 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T18:14:16.404 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T18:14:16.405 INFO:teuthology.orchestra.run.smithi150.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T18:14:16.405 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T18:14:16.405 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T18:14:16.405 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T18:14:16.405 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T18:14:16.405 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T18:14:16.405 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T18:14:16.405 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T18:14:16.405 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T18:14:16.405 INFO:teuthology.orchestra.run.smithi150.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T18:14:16.405 INFO:teuthology.orchestra.run.smithi150.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T18:14:16.405 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T18:14:16.405 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T18:14:16.405 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T18:14:16.406 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:16.406 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T18:14:16.406 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:14:16.406 INFO:teuthology.orchestra.run.smithi150.stdout:Install 34 Packages 2024-09-10T18:14:16.406 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:16.407 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 27 M 2024-09-10T18:14:16.407 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 96 M 2024-09-10T18:14:16.407 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-10T18:14:16.753 INFO:teuthology.orchestra.run.smithi150.stdout:(1/34): ceph-17.2.7-1606.g5be57395.el9.x86_64.r 51 kB/s | 6.5 kB 00:00 2024-09-10T18:14:16.934 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 4/4 2024-09-10T18:14:16.935 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 1/4 2024-09-10T18:14:16.935 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 2/4 2024-09-10T18:14:16.935 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-10T18:14:16.936 INFO:teuthology.orchestra.run.smithi150.stdout:(2/34): ceph-mgr-17.2.7-1606.g5be57395.el9.x86_ 4.5 MB/s | 1.4 MB 00:00 2024-09-10T18:14:16.987 INFO:teuthology.orchestra.run.smithi150.stdout:(3/34): ceph-mds-17.2.7-1606.g5be57395.el9.x86_ 5.7 MB/s | 2.1 MB 00:00 2024-09-10T18:14:17.079 INFO:teuthology.orchestra.run.smithi150.stdout:(4/34): ceph-mgr-modules-core-17.2.7-1606.g5be5 2.6 MB/s | 240 kB 00:00 2024-09-10T18:14:17.162 INFO:teuthology.orchestra.run.smithi150.stdout:(5/34): ceph-mon-17.2.7-1606.g5be57395.el9.x86_ 9.6 MB/s | 3.9 MB 00:00 2024-09-10T18:14:17.263 INFO:teuthology.orchestra.run.smithi150.stdout:(6/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-09-10T18:14:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-10T18:14:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:17.274 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T18:14:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T18:14:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T18:14:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:17.275 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T18:14:17.438 INFO:teuthology.orchestra.run.smithi150.stdout:(7/34): ceph-osd-17.2.7-1606.g5be57395.el9.x86_ 31 MB/s | 15 MB 00:00 2024-09-10T18:14:17.463 INFO:teuthology.orchestra.run.smithi150.stdout:(8/34): python3-ply-3.11-14.el9.noarch.rpm 532 kB/s | 106 kB 00:00 2024-09-10T18:14:17.522 INFO:teuthology.orchestra.run.smithi150.stdout:(9/34): python3-requests-2.25.1-8.el9.noarch.rp 2.1 MB/s | 125 kB 00:00 2024-09-10T18:14:17.580 INFO:teuthology.orchestra.run.smithi150.stdout:(10/34): python3-urllib3-1.26.5-6.el9.noarch.rp 3.6 MB/s | 215 kB 00:00 2024-09-10T18:14:17.587 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph 2024-09-10T18:14:17.606 INFO:teuthology.orchestra.run.smithi150.stdout:(11/34): python3-pycparser-2.20-6.el9.noarch.rp 808 kB/s | 135 kB 00:00 2024-09-10T18:14:17.639 INFO:teuthology.orchestra.run.smithi150.stdout:(12/34): python3-cryptography-36.0.1-4.el9.x86_ 2.6 MB/s | 1.2 MB 00:00 2024-09-10T18:14:17.690 INFO:teuthology.orchestra.run.smithi150.stdout:(13/34): python3-markupsafe-1.1.1-12.el9.x86_64 416 kB/s | 35 kB 00:00 2024-09-10T18:14:17.723 INFO:teuthology.orchestra.run.smithi150.stdout:(14/34): python3-packaging-20.9-5.el9.noarch.rp 923 kB/s | 77 kB 00:00 2024-09-10T18:14:17.765 INFO:teuthology.orchestra.run.smithi150.stdout:(15/34): python3-mako-1.1.4-6.el9.noarch.rpm 934 kB/s | 172 kB 00:00 2024-09-10T18:14:17.782 INFO:teuthology.orchestra.run.smithi150.stdout:(16/34): python3-toml-0.10.2-6.el9.noarch.rpm 713 kB/s | 42 kB 00:00 2024-09-10T18:14:17.799 INFO:teuthology.orchestra.run.smithi150.stdout:(17/34): python3-pytz-2021.1-5.el9.noarch.rpm 467 kB/s | 51 kB 00:00 2024-09-10T18:14:17.841 INFO:teuthology.orchestra.run.smithi150.stdout:(18/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 575 kB/s | 43 kB 00:00 2024-09-10T18:14:17.866 INFO:teuthology.orchestra.run.smithi150.stdout:(19/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 464 kB/s | 11 kB 00:00 2024-09-10T18:14:17.891 INFO:teuthology.orchestra.run.smithi150.stdout:(20/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.5 MB/s | 172 kB 00:00 2024-09-10T18:14:17.916 INFO:teuthology.orchestra.run.smithi150.stdout:(21/34): python3-jaraco-classes-3.2.1-5.el9.noa 353 kB/s | 18 kB 00:00 2024-09-10T18:14:17.942 INFO:teuthology.orchestra.run.smithi150.stdout:(22/34): python3-jaraco-collections-3.0.0-8.el9 932 kB/s | 23 kB 00:00 2024-09-10T18:14:17.967 INFO:teuthology.orchestra.run.smithi150.stdout:(23/34): python3-jaraco-functools-3.5.0-2.el9.n 388 kB/s | 19 kB 00:00 2024-09-10T18:14:17.993 INFO:teuthology.orchestra.run.smithi150.stdout:(24/34): python3-jaraco-text-3.2.0-6.el9.noarch 390 kB/s | 20 kB 00:00 2024-09-10T18:14:18.018 INFO:teuthology.orchestra.run.smithi150.stdout:(25/34): python3-logutils-0.3.5-21.el9.noarch.r 920 kB/s | 46 kB 00:00 2024-09-10T18:14:18.051 INFO:teuthology.orchestra.run.smithi150.stdout:(26/34): python3-cherrypy-18.6.1-2.el9.noarch.r 1.5 MB/s | 358 kB 00:00 2024-09-10T18:14:18.077 INFO:teuthology.orchestra.run.smithi150.stdout:(27/34): python3-more-itertools-8.12.0-2.el9.no 942 kB/s | 79 kB 00:00 2024-09-10T18:14:18.102 INFO:teuthology.orchestra.run.smithi150.stdout:(28/34): python3-portend-3.1.0-2.el9.noarch.rpm 661 kB/s | 16 kB 00:00 2024-09-10T18:14:18.127 INFO:teuthology.orchestra.run.smithi150.stdout:(29/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-09-10T18:14:18.153 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:38 ago on Tue 10 Sep 2024 06:13:40 PM UTC. 2024-09-10T18:14:18.153 INFO:teuthology.orchestra.run.smithi150.stdout:(30/34): python3-tempora-5.0.0-2.el9.noarch.rpm 709 kB/s | 36 kB 00:00 2024-09-10T18:14:18.178 INFO:teuthology.orchestra.run.smithi150.stdout:(31/34): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-09-10T18:14:18.203 INFO:teuthology.orchestra.run.smithi150.stdout:(32/34): python3-zc-lockfile-2.0-10.el9.noarch. 800 kB/s | 20 kB 00:00 2024-09-10T18:14:18.228 INFO:teuthology.orchestra.run.smithi150.stdout:(33/34): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-10T18:14:18.253 INFO:teuthology.orchestra.run.smithi150.stdout:(34/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-10T18:14:18.258 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:14:18.258 INFO:teuthology.orchestra.run.smithi150.stdout:Total 15 MB/s | 27 MB 00:01 2024-09-10T18:14:18.288 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T18:14:18.290 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:14:18.290 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-10T18:14:18.290 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:14:18.290 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T18:14:18.290 INFO:teuthology.orchestra.run.smithi115.stdout: ceph x86_64 2:17.2.7-1606.g5be57395.el9 ceph 6.5 k 2024-09-10T18:14:18.290 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-10T18:14:18.291 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mds x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.1 M 2024-09-10T18:14:18.291 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr x86_64 2:17.2.7-1606.g5be57395.el9 ceph 1.4 M 2024-09-10T18:14:18.291 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 240 k 2024-09-10T18:14:18.291 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mon x86_64 2:17.2.7-1606.g5be57395.el9 ceph 3.9 M 2024-09-10T18:14:18.291 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-osd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 15 M 2024-09-10T18:14:18.291 INFO:teuthology.orchestra.run.smithi115.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T18:14:18.291 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T18:14:18.291 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T18:14:18.291 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T18:14:18.291 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T18:14:18.291 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T18:14:18.291 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T18:14:18.291 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-collections 2024-09-10T18:14:18.292 INFO:teuthology.orchestra.run.smithi115.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-10T18:14:18.292 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T18:14:18.292 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T18:14:18.292 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T18:14:18.292 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T18:14:18.292 INFO:teuthology.orchestra.run.smithi115.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T18:14:18.292 INFO:teuthology.orchestra.run.smithi115.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T18:14:18.292 INFO:teuthology.orchestra.run.smithi115.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T18:14:18.292 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T18:14:18.292 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T18:14:18.292 INFO:teuthology.orchestra.run.smithi115.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T18:14:18.292 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T18:14:18.293 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T18:14:18.293 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T18:14:18.293 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T18:14:18.293 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T18:14:18.293 INFO:teuthology.orchestra.run.smithi115.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T18:14:18.293 INFO:teuthology.orchestra.run.smithi115.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T18:14:18.293 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T18:14:18.293 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T18:14:18.293 INFO:teuthology.orchestra.run.smithi115.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T18:14:18.293 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:18.293 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T18:14:18.293 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:14:18.293 INFO:teuthology.orchestra.run.smithi115.stdout:Install 34 Packages 2024-09-10T18:14:18.293 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:18.294 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 27 M 2024-09-10T18:14:18.294 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 96 M 2024-09-10T18:14:18.294 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T18:14:18.391 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T18:14:18.429 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T18:14:18.429 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T18:14:18.792 INFO:teuthology.orchestra.run.smithi115.stdout:(1/34): ceph-17.2.7-1606.g5be57395.el9.x86_64.r 55 kB/s | 6.5 kB 00:00 2024-09-10T18:14:18.804 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T18:14:18.804 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T18:14:19.009 INFO:teuthology.orchestra.run.smithi115.stdout:(2/34): ceph-mgr-17.2.7-1606.g5be57395.el9.x86_ 4.2 MB/s | 1.4 MB 00:00 2024-09-10T18:14:19.118 INFO:teuthology.orchestra.run.smithi115.stdout:(3/34): ceph-mds-17.2.7-1606.g5be57395.el9.x86_ 4.7 MB/s | 2.1 MB 00:00 2024-09-10T18:14:19.168 INFO:teuthology.orchestra.run.smithi115.stdout:(4/34): ceph-mgr-modules-core-17.2.7-1606.g5be5 4.7 MB/s | 240 kB 00:00 2024-09-10T18:14:19.360 INFO:teuthology.orchestra.run.smithi115.stdout:(5/34): ceph-mon-17.2.7-1606.g5be57395.el9.x86_ 6.9 MB/s | 3.9 MB 00:00 2024-09-10T18:14:19.577 INFO:teuthology.orchestra.run.smithi115.stdout:(6/34): ceph-osd-17.2.7-1606.g5be57395.el9.x86_ 27 MB/s | 15 MB 00:00 2024-09-10T18:14:20.045 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T18:14:20.106 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-10T18:14:20.138 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-10T18:14:20.152 INFO:teuthology.orchestra.run.smithi115.stdout:(7/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 257 kB/s | 253 kB 00:00 2024-09-10T18:14:20.171 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-10T18:14:20.244 INFO:teuthology.orchestra.run.smithi115.stdout:(8/34): python3-ply-3.11-14.el9.noarch.rpm 161 kB/s | 106 kB 00:00 2024-09-10T18:14:20.263 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-10T18:14:20.341 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-10T18:14:20.383 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-10T18:14:20.417 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-10T18:14:20.447 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-10T18:14:20.499 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-10T18:14:20.528 INFO:teuthology.orchestra.run.smithi115.stdout:(9/34): python3-cryptography-36.0.1-4.el9.x86_6 1.1 MB/s | 1.2 MB 00:01 2024-09-10T18:14:20.531 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-10T18:14:20.581 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-10T18:14:20.615 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-10T18:14:20.650 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-10T18:14:20.698 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-10T18:14:20.735 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-10T18:14:20.812 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-10T18:14:20.870 INFO:teuthology.orchestra.run.smithi115.stdout:(10/34): python3-pycparser-2.20-6.el9.noarch.rp 188 kB/s | 135 kB 00:00 2024-09-10T18:14:20.920 INFO:teuthology.orchestra.run.smithi115.stdout:(11/34): python3-requests-2.25.1-8.el9.noarch.r 185 kB/s | 125 kB 00:00 2024-09-10T18:14:20.953 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-10T18:14:21.004 INFO:teuthology.orchestra.run.smithi115.stdout:(12/34): python3-markupsafe-1.1.1-12.el9.x86_64 417 kB/s | 35 kB 00:00 2024-09-10T18:14:21.039 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-10T18:14:21.054 INFO:teuthology.orchestra.run.smithi115.stdout:(13/34): python3-mako-1.1.4-6.el9.noarch.rpm 939 kB/s | 172 kB 00:00 2024-09-10T18:14:21.092 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-10T18:14:21.095 INFO:teuthology.orchestra.run.smithi115.stdout:(14/34): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-09-10T18:14:21.121 INFO:teuthology.orchestra.run.smithi115.stdout:(15/34): python3-packaging-20.9-5.el9.noarch.rp 659 kB/s | 77 kB 00:00 2024-09-10T18:14:21.130 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-10T18:14:21.146 INFO:teuthology.orchestra.run.smithi115.stdout:(16/34): python3-toml-0.10.2-6.el9.noarch.rpm 826 kB/s | 42 kB 00:00 2024-09-10T18:14:21.179 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-10T18:14:21.188 INFO:teuthology.orchestra.run.smithi115.stdout:(17/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 650 kB/s | 43 kB 00:00 2024-09-10T18:14:21.222 INFO:teuthology.orchestra.run.smithi115.stdout:(18/34): python3-cherrypy-18.6.1-2.el9.noarch.r 11 MB/s | 358 kB 00:00 2024-09-10T18:14:21.234 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-10T18:14:21.247 INFO:teuthology.orchestra.run.smithi115.stdout:(19/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-09-10T18:14:21.272 INFO:teuthology.orchestra.run.smithi115.stdout:(20/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 212 kB/s | 11 kB 00:00 2024-09-10T18:14:21.297 INFO:teuthology.orchestra.run.smithi115.stdout:(21/34): python3-jaraco-classes-3.2.1-5.el9.noa 352 kB/s | 18 kB 00:00 2024-09-10T18:14:21.323 INFO:teuthology.orchestra.run.smithi115.stdout:(22/34): python3-jaraco-collections-3.0.0-8.el9 462 kB/s | 23 kB 00:00 2024-09-10T18:14:21.348 INFO:teuthology.orchestra.run.smithi115.stdout:(23/34): python3-urllib3-1.26.5-6.el9.noarch.rp 262 kB/s | 215 kB 00:00 2024-09-10T18:14:21.373 INFO:teuthology.orchestra.run.smithi115.stdout:(24/34): python3-jaraco-functools-3.5.0-2.el9.n 258 kB/s | 19 kB 00:00 2024-09-10T18:14:21.399 INFO:teuthology.orchestra.run.smithi115.stdout:(25/34): python3-jaraco-text-3.2.0-6.el9.noarch 261 kB/s | 20 kB 00:00 2024-09-10T18:14:21.432 INFO:teuthology.orchestra.run.smithi115.stdout:(26/34): python3-logutils-0.3.5-21.el9.noarch.r 791 kB/s | 46 kB 00:00 2024-09-10T18:14:21.445 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-10T18:14:21.457 INFO:teuthology.orchestra.run.smithi115.stdout:(27/34): python3-portend-3.1.0-2.el9.noarch.rpm 712 kB/s | 16 kB 00:00 2024-09-10T18:14:21.483 INFO:teuthology.orchestra.run.smithi115.stdout:(28/34): python3-more-itertools-8.12.0-2.el9.no 723 kB/s | 79 kB 00:00 2024-09-10T18:14:21.492 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-10T18:14:21.516 INFO:teuthology.orchestra.run.smithi115.stdout:(29/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.5 MB/s | 90 kB 00:00 2024-09-10T18:14:21.542 INFO:teuthology.orchestra.run.smithi115.stdout:(30/34): python3-tempora-5.0.0-2.el9.noarch.rpm 612 kB/s | 36 kB 00:00 2024-09-10T18:14:21.567 INFO:teuthology.orchestra.run.smithi115.stdout:(31/34): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-09-10T18:14:21.570 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-10T18:14:21.592 INFO:teuthology.orchestra.run.smithi115.stdout:(32/34): python3-zc-lockfile-2.0-10.el9.noarch. 806 kB/s | 20 kB 00:00 2024-09-10T18:14:21.617 INFO:teuthology.orchestra.run.smithi115.stdout:(33/34): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-10T18:14:21.642 INFO:teuthology.orchestra.run.smithi115.stdout:(34/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-10T18:14:21.648 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:14:21.648 INFO:teuthology.orchestra.run.smithi115.stdout:Total 8.1 MB/s | 27 MB 00:03 2024-09-10T18:14:21.659 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-10T18:14:21.703 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-10T18:14:21.778 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T18:14:21.816 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T18:14:21.816 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T18:14:21.822 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-10T18:14:21.978 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9. 29/34 2024-09-10T18:14:22.002 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 30/34 2024-09-10T18:14:22.027 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 30/34 2024-09-10T18:14:22.027 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:14:22.027 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T18:14:22.027 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T18:14:22.027 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T18:14:22.027 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:22.187 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T18:14:22.187 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T18:14:23.336 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 31/34 2024-09-10T18:14:23.366 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 31/34 2024-09-10T18:14:23.366 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:14:23.366 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T18:14:23.366 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T18:14:23.366 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T18:14:23.366 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:23.414 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T18:14:23.487 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-10T18:14:23.519 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-10T18:14:23.552 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-10T18:14:23.645 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-10T18:14:23.724 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-10T18:14:23.755 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 32/34 2024-09-10T18:14:23.765 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-10T18:14:23.794 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 32/34 2024-09-10T18:14:23.794 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:14:23.794 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T18:14:23.794 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T18:14:23.794 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T18:14:23.794 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:23.800 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-10T18:14:23.830 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-10T18:14:23.856 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-10T18:14:23.889 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-10T18:14:23.930 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-10T18:14:23.965 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-10T18:14:23.999 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-10T18:14:24.005 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 33/34 2024-09-10T18:14:24.030 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 33/34 2024-09-10T18:14:24.030 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:14:24.030 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T18:14:24.030 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T18:14:24.031 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T18:14:24.031 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:24.047 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-10T18:14:24.089 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-10T18:14:24.094 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-2:17.2.7-1606.g5be57395.el9.x86_64 34/34 2024-09-10T18:14:24.160 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-10T18:14:24.299 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-10T18:14:24.387 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-10T18:14:24.441 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-10T18:14:24.479 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-10T18:14:24.528 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-10T18:14:24.583 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-10T18:14:24.768 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-10T18:14:24.826 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-10T18:14:24.894 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-10T18:14:24.985 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-10T18:14:25.032 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-10T18:14:25.149 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-10T18:14:25.295 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9. 29/34 2024-09-10T18:14:25.327 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 30/34 2024-09-10T18:14:25.352 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 30/34 2024-09-10T18:14:25.353 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:14:25.353 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T18:14:25.353 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T18:14:25.353 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T18:14:25.353 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:25.754 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-2:17.2.7-1606.g5be57395.el9.x86_64 34/34 2024-09-10T18:14:25.754 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-2:17.2.7-1606.g5be57395.el9.x86_64 1/34 2024-09-10T18:14:25.754 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 2/34 2024-09-10T18:14:25.754 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 3/34 2024-09-10T18:14:25.754 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 4/34 2024-09-10T18:14:25.754 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 5/34 2024-09-10T18:14:25.754 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9. 6/34 2024-09-10T18:14:25.754 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-10T18:14:25.755 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-10T18:14:25.755 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-10T18:14:25.755 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-10T18:14:25.755 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-10T18:14:25.755 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-10T18:14:25.755 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-10T18:14:25.755 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-10T18:14:25.755 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-10T18:14:25.755 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-10T18:14:25.755 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-10T18:14:25.755 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-10T18:14:25.755 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-10T18:14:25.755 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-10T18:14:25.755 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-10T18:14:25.756 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-10T18:14:25.756 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-10T18:14:25.756 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-10T18:14:25.756 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-10T18:14:25.757 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-10T18:14:25.757 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-10T18:14:25.757 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-10T18:14:25.757 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-10T18:14:25.757 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-10T18:14:25.757 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-10T18:14:25.757 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-10T18:14:25.757 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-10T18:14:26.584 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-10T18:14:26.584 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:26.584 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-10T18:14:26.585 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:26.585 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:26.585 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:26.585 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:14:26.585 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:26.585 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:26.585 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T18:14:26.585 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T18:14:26.585 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T18:14:26.585 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T18:14:26.585 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T18:14:26.586 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T18:14:26.586 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T18:14:26.586 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T18:14:26.586 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T18:14:26.586 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T18:14:26.586 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T18:14:26.586 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T18:14:26.586 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T18:14:26.586 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T18:14:26.586 INFO:teuthology.orchestra.run.smithi150.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T18:14:26.586 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T18:14:26.587 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T18:14:26.587 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T18:14:26.587 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T18:14:26.587 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T18:14:26.587 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T18:14:26.587 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T18:14:26.587 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T18:14:26.587 INFO:teuthology.orchestra.run.smithi150.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T18:14:26.587 INFO:teuthology.orchestra.run.smithi150.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T18:14:26.587 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T18:14:26.587 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T18:14:26.588 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T18:14:26.588 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:26.588 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T18:14:26.728 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 31/34 2024-09-10T18:14:26.757 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 31/34 2024-09-10T18:14:26.757 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:14:26.757 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T18:14:26.757 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T18:14:26.757 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T18:14:26.757 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:26.925 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-base 2024-09-10T18:14:27.129 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 32/34 2024-09-10T18:14:27.153 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 32/34 2024-09-10T18:14:27.153 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:14:27.154 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T18:14:27.154 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T18:14:27.154 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T18:14:27.154 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:27.355 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 33/34 2024-09-10T18:14:27.379 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 33/34 2024-09-10T18:14:27.379 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:14:27.379 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T18:14:27.379 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T18:14:27.379 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T18:14:27.380 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:27.444 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-2:17.2.7-1606.g5be57395.el9.x86_64 34/34 2024-09-10T18:14:27.495 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:49 ago on Tue 10 Sep 2024 06:13:38 PM UTC. 2024-09-10T18:14:27.584 INFO:teuthology.orchestra.run.smithi150.stdout:Package ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:14:27.627 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T18:14:27.628 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-09-10T18:14:27.628 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T18:14:27.693 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install cephadm 2024-09-10T18:14:28.256 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:50 ago on Tue 10 Sep 2024 06:13:38 PM UTC. 2024-09-10T18:14:28.388 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T18:14:28.389 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:14:28.389 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-09-10T18:14:28.389 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:14:28.389 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-10T18:14:28.389 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 83 k 2024-09-10T18:14:28.389 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:28.389 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T18:14:28.389 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:14:28.389 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-09-10T18:14:28.389 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:28.389 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 83 k 2024-09-10T18:14:28.390 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 375 k 2024-09-10T18:14:28.390 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-10T18:14:28.578 INFO:teuthology.orchestra.run.smithi150.stdout:cephadm-17.2.7-1606.g5be57395.el9.noarch.rpm 439 kB/s | 83 kB 00:00 2024-09-10T18:14:28.579 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:14:28.579 INFO:teuthology.orchestra.run.smithi150.stdout:Total 436 kB/s | 83 kB 00:00 2024-09-10T18:14:28.579 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T18:14:28.584 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T18:14:28.584 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T18:14:28.586 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T18:14:28.586 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T18:14:28.652 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T18:14:29.078 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:14:29.114 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-2:17.2.7-1606.g5be57395.el9.x86_64 34/34 2024-09-10T18:14:29.114 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-2:17.2.7-1606.g5be57395.el9.x86_64 1/34 2024-09-10T18:14:29.114 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 2/34 2024-09-10T18:14:29.114 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 3/34 2024-09-10T18:14:29.114 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 4/34 2024-09-10T18:14:29.114 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 5/34 2024-09-10T18:14:29.114 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9. 6/34 2024-09-10T18:14:29.114 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-10T18:14:29.114 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-10T18:14:29.114 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-10T18:14:29.115 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-10T18:14:29.115 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-10T18:14:29.115 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-10T18:14:29.115 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-10T18:14:29.115 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-10T18:14:29.115 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-10T18:14:29.115 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-10T18:14:29.115 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-10T18:14:29.115 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-10T18:14:29.115 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-10T18:14:29.115 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-10T18:14:29.115 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-10T18:14:29.116 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-10T18:14:29.116 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-10T18:14:29.116 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-10T18:14:29.116 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-10T18:14:29.116 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-10T18:14:29.116 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-10T18:14:29.117 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-10T18:14:29.117 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-10T18:14:29.117 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-10T18:14:29.117 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-10T18:14:29.117 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-10T18:14:29.117 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-10T18:14:29.162 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:14:29.499 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:14:29.749 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:14:29.749 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:29.749 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-10T18:14:29.749 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:14:29.750 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:29.750 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T18:14:29.939 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-immutable-object-cache 2024-09-10T18:14:29.972 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-10T18:14:29.972 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:29.973 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T18:14:29.973 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:29.973 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:29.973 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:29.973 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:14:29.973 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:29.973 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:29.973 INFO:teuthology.orchestra.run.smithi115.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T18:14:29.973 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T18:14:29.973 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T18:14:29.973 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T18:14:29.973 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T18:14:29.973 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T18:14:29.974 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T18:14:29.974 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T18:14:29.974 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T18:14:29.974 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T18:14:29.974 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T18:14:29.974 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T18:14:29.974 INFO:teuthology.orchestra.run.smithi115.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T18:14:29.974 INFO:teuthology.orchestra.run.smithi115.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T18:14:29.974 INFO:teuthology.orchestra.run.smithi115.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T18:14:29.974 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T18:14:29.974 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T18:14:29.974 INFO:teuthology.orchestra.run.smithi115.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T18:14:29.974 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T18:14:29.975 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T18:14:29.975 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T18:14:29.975 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T18:14:29.975 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T18:14:29.975 INFO:teuthology.orchestra.run.smithi115.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T18:14:29.975 INFO:teuthology.orchestra.run.smithi115.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T18:14:29.975 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T18:14:29.975 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T18:14:29.975 INFO:teuthology.orchestra.run.smithi115.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T18:14:29.975 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:29.975 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T18:14:30.350 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-base 2024-09-10T18:14:30.517 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:52 ago on Tue 10 Sep 2024 06:13:38 PM UTC. 2024-09-10T18:14:30.655 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T18:14:30.655 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:14:30.655 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repo Size 2024-09-10T18:14:30.655 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:14:30.655 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-10T18:14:30.655 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1606.g5be57395.el9 ceph 137 k 2024-09-10T18:14:30.655 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:30.656 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T18:14:30.656 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:14:30.656 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-09-10T18:14:30.656 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:30.656 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 137 k 2024-09-10T18:14:30.656 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 379 k 2024-09-10T18:14:30.657 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-10T18:14:30.853 INFO:teuthology.orchestra.run.smithi150.stdout:ceph-immutable-object-cache-17.2.7-1606.g5be573 703 kB/s | 137 kB 00:00 2024-09-10T18:14:30.854 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:14:30.854 INFO:teuthology.orchestra.run.smithi150.stdout:Total 697 kB/s | 137 kB 00:00 2024-09-10T18:14:30.854 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T18:14:30.861 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T18:14:30.861 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T18:14:30.912 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:50 ago on Tue 10 Sep 2024 06:13:40 PM UTC. 2024-09-10T18:14:30.925 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T18:14:30.925 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T18:14:31.003 INFO:teuthology.orchestra.run.smithi115.stdout:Package ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:14:31.045 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T18:14:31.046 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-10T18:14:31.046 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T18:14:31.080 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T18:14:31.121 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install cephadm 2024-09-10T18:14:31.170 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1606.g5be57395. 1/1 2024-09-10T18:14:31.196 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1606.g5be57395. 1/1 2024-09-10T18:14:31.196 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:14:31.197 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T18:14:31.197 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:31.686 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:51 ago on Tue 10 Sep 2024 06:13:40 PM UTC. 2024-09-10T18:14:31.831 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T18:14:31.831 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:14:31.832 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-10T18:14:31.832 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:14:31.832 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T18:14:31.832 INFO:teuthology.orchestra.run.smithi115.stdout: cephadm noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 83 k 2024-09-10T18:14:31.832 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:31.832 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T18:14:31.832 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:14:31.832 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-10T18:14:31.832 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:31.832 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 83 k 2024-09-10T18:14:31.832 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 375 k 2024-09-10T18:14:31.832 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T18:14:32.012 INFO:teuthology.orchestra.run.smithi115.stdout:cephadm-17.2.7-1606.g5be57395.el9.noarch.rpm 463 kB/s | 83 kB 00:00 2024-09-10T18:14:32.012 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:14:32.012 INFO:teuthology.orchestra.run.smithi115.stdout:Total 458 kB/s | 83 kB 00:00 2024-09-10T18:14:32.012 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T18:14:32.018 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T18:14:32.018 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T18:14:32.019 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T18:14:32.020 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T18:14:32.087 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T18:14:32.108 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1606.g5be57395. 1/1 2024-09-10T18:14:32.108 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:32.108 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-10T18:14:32.108 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:32.109 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:32.109 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T18:14:32.313 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-mgr 2024-09-10T18:14:32.510 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:14:32.607 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:14:32.887 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:54 ago on Tue 10 Sep 2024 06:13:38 PM UTC. 2024-09-10T18:14:32.923 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:14:32.978 INFO:teuthology.orchestra.run.smithi150.stdout:Package ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:14:33.019 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T18:14:33.020 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-09-10T18:14:33.020 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T18:14:33.092 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-mgr-dashboard 2024-09-10T18:14:33.180 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:14:33.180 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:33.180 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T18:14:33.180 INFO:teuthology.orchestra.run.smithi115.stdout: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:14:33.180 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:33.181 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T18:14:33.383 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-immutable-object-cache 2024-09-10T18:14:33.658 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:55 ago on Tue 10 Sep 2024 06:13:38 PM UTC. 2024-09-10T18:14:33.791 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T18:14:33.793 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:14:33.793 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-09-10T18:14:33.793 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:14:33.793 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-10T18:14:33.793 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 3.4 M 2024-09-10T18:14:33.794 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-09-10T18:14:33.794 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 24 k 2024-09-10T18:14:33.794 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 15 k 2024-09-10T18:14:33.794 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-10T18:14:33.794 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T18:14:33.794 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T18:14:33.794 INFO:teuthology.orchestra.run.smithi150.stdout:Installing weak dependencies: 2024-09-10T18:14:33.794 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-10T18:14:33.794 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:33.794 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T18:14:33.794 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:14:33.794 INFO:teuthology.orchestra.run.smithi150.stdout:Install 7 Packages 2024-09-10T18:14:33.795 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:33.795 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 3.7 M 2024-09-10T18:14:33.795 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 80 M 2024-09-10T18:14:33.795 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-10T18:14:33.964 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:53 ago on Tue 10 Sep 2024 06:13:40 PM UTC. 2024-09-10T18:14:34.100 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T18:14:34.100 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:14:34.100 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repo Size 2024-09-10T18:14:34.100 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:14:34.100 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T18:14:34.101 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1606.g5be57395.el9 ceph 137 k 2024-09-10T18:14:34.101 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:34.101 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T18:14:34.101 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:14:34.101 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-10T18:14:34.101 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:34.101 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 137 k 2024-09-10T18:14:34.102 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 379 k 2024-09-10T18:14:34.102 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T18:14:34.123 INFO:teuthology.orchestra.run.smithi150.stdout:(1/7): ceph-prometheus-alerts-17.2.7-1606.g5be5 100 kB/s | 15 kB 00:00 2024-09-10T18:14:34.148 INFO:teuthology.orchestra.run.smithi150.stdout:(2/7): ceph-grafana-dashboards-17.2.7-1606.g5be 143 kB/s | 24 kB 00:00 2024-09-10T18:14:34.190 INFO:teuthology.orchestra.run.smithi150.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 135 kB/s | 9.0 kB 00:00 2024-09-10T18:14:34.215 INFO:teuthology.orchestra.run.smithi150.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-10T18:14:34.241 INFO:teuthology.orchestra.run.smithi150.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 444 kB/s | 41 kB 00:00 2024-09-10T18:14:34.266 INFO:teuthology.orchestra.run.smithi150.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-10T18:14:34.294 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-immutable-object-cache-17.2.7-1606.g5be573 714 kB/s | 137 kB 00:00 2024-09-10T18:14:34.295 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:14:34.295 INFO:teuthology.orchestra.run.smithi115.stdout:Total 709 kB/s | 137 kB 00:00 2024-09-10T18:14:34.295 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T18:14:34.302 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T18:14:34.303 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T18:14:34.333 INFO:teuthology.orchestra.run.smithi150.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1606.g5be57395 9.4 MB/s | 3.4 MB 00:00 2024-09-10T18:14:34.336 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:14:34.336 INFO:teuthology.orchestra.run.smithi150.stdout:Total 6.7 MB/s | 3.7 MB 00:00 2024-09-10T18:14:34.336 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T18:14:34.344 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T18:14:34.344 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T18:14:34.363 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T18:14:34.364 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T18:14:34.432 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T18:14:34.432 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T18:14:34.524 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T18:14:34.570 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1606.g5be57395. 1/1 2024-09-10T18:14:34.595 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1606.g5be57395. 1/1 2024-09-10T18:14:34.595 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:14:34.595 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T18:14:34.595 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:34.740 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T18:14:34.792 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-10T18:14:34.826 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-10T18:14:34.873 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-10T18:14:34.905 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-10T18:14:34.933 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.n 5/7 2024-09-10T18:14:35.454 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1606.g5be57395. 1/1 2024-09-10T18:14:35.454 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:35.454 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T18:14:35.454 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-immutable-object-cache-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:14:35.455 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:35.455 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T18:14:35.696 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr 2024-09-10T18:14:35.916 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9. 6/7 2024-09-10T18:14:35.954 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarc 7/7 2024-09-10T18:14:36.275 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:56 ago on Tue 10 Sep 2024 06:13:40 PM UTC. 2024-09-10T18:14:36.368 INFO:teuthology.orchestra.run.smithi115.stdout:Package ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:14:36.411 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T18:14:36.412 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-10T18:14:36.412 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T18:14:36.485 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr-dashboard 2024-09-10T18:14:36.948 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarc 7/7 2024-09-10T18:14:36.948 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9. 1/7 2024-09-10T18:14:36.948 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarc 2/7 2024-09-10T18:14:36.948 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.n 3/7 2024-09-10T18:14:36.948 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-10T18:14:36.948 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-10T18:14:36.948 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-10T18:14:37.053 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:57 ago on Tue 10 Sep 2024 06:13:40 PM UTC. 2024-09-10T18:14:37.184 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T18:14:37.185 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:14:37.185 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-10T18:14:37.185 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:14:37.185 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T18:14:37.186 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 3.4 M 2024-09-10T18:14:37.186 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-10T18:14:37.186 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 24 k 2024-09-10T18:14:37.186 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 15 k 2024-09-10T18:14:37.186 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-10T18:14:37.186 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T18:14:37.186 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T18:14:37.187 INFO:teuthology.orchestra.run.smithi115.stdout:Installing weak dependencies: 2024-09-10T18:14:37.187 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-10T18:14:37.187 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:37.187 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T18:14:37.187 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:14:37.188 INFO:teuthology.orchestra.run.smithi115.stdout:Install 7 Packages 2024-09-10T18:14:37.188 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:37.189 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 3.7 M 2024-09-10T18:14:37.189 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 80 M 2024-09-10T18:14:37.189 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T18:14:37.246 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-10T18:14:37.246 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:37.246 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-10T18:14:37.246 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:14:37.246 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:14:37.246 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:14:37.246 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-10T18:14:37.247 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-10T18:14:37.247 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T18:14:37.247 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T18:14:37.247 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:37.247 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T18:14:37.381 INFO:teuthology.orchestra.run.smithi115.stdout:(1/7): ceph-prometheus-alerts-17.2.7-1606.g5be5 91 kB/s | 15 kB 00:00 2024-09-10T18:14:37.406 INFO:teuthology.orchestra.run.smithi115.stdout:(2/7): ceph-grafana-dashboards-17.2.7-1606.g5be 131 kB/s | 24 kB 00:00 2024-09-10T18:14:37.449 INFO:teuthology.orchestra.run.smithi115.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 135 kB/s | 9.0 kB 00:00 2024-09-10T18:14:37.469 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-10T18:14:37.473 INFO:teuthology.orchestra.run.smithi115.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-10T18:14:37.499 INFO:teuthology.orchestra.run.smithi115.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 443 kB/s | 41 kB 00:00 2024-09-10T18:14:37.524 INFO:teuthology.orchestra.run.smithi115.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-10T18:14:37.608 INFO:teuthology.orchestra.run.smithi115.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1606.g5be57395 8.6 MB/s | 3.4 MB 00:00 2024-09-10T18:14:37.611 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:14:37.611 INFO:teuthology.orchestra.run.smithi115.stdout:Total 8.6 MB/s | 3.7 MB 00:00 2024-09-10T18:14:37.612 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T18:14:37.620 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T18:14:37.621 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T18:14:37.704 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T18:14:37.704 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T18:14:38.009 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T18:14:38.029 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:00 ago on Tue 10 Sep 2024 06:13:38 PM UTC. 2024-09-10T18:14:38.084 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-10T18:14:38.118 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-10T18:14:38.161 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T18:14:38.162 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-09-10T18:14:38.162 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-09-10T18:14:38.162 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-09-10T18:14:38.162 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-10T18:14:38.163 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 7.4 M 2024-09-10T18:14:38.163 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-09-10T18:14:38.163 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T18:14:38.163 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T18:14:38.163 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T18:14:38.163 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T18:14:38.163 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T18:14:38.163 INFO:teuthology.orchestra.run.smithi150.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T18:14:38.163 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T18:14:38.163 INFO:teuthology.orchestra.run.smithi150.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T18:14:38.163 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T18:14:38.163 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T18:14:38.163 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T18:14:38.163 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:38.164 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T18:14:38.164 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-09-10T18:14:38.164 INFO:teuthology.orchestra.run.smithi150.stdout:Install 12 Packages 2024-09-10T18:14:38.164 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:38.164 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 38 M 2024-09-10T18:14:38.164 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 212 M 2024-09-10T18:14:38.164 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-10T18:14:38.165 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-10T18:14:38.205 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-10T18:14:38.233 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.n 5/7 2024-09-10T18:14:38.582 INFO:teuthology.orchestra.run.smithi150.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 640 kB/s | 191 kB 00:00 2024-09-10T18:14:38.724 INFO:teuthology.orchestra.run.smithi150.stdout:(2/12): ceph-mgr-diskprediction-local-17.2.7-16 17 MB/s | 7.4 MB 00:00 2024-09-10T18:14:38.749 INFO:teuthology.orchestra.run.smithi150.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 196 kB/s | 33 kB 00:00 2024-09-10T18:14:38.783 INFO:teuthology.orchestra.run.smithi150.stdout:(4/12): libgfortran-11.5.0-2.el9.x86_64.rpm 1.6 MB/s | 801 kB 00:00 2024-09-10T18:14:38.816 INFO:teuthology.orchestra.run.smithi150.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 272 kB/s | 18 kB 00:00 2024-09-10T18:14:38.925 INFO:teuthology.orchestra.run.smithi150.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 271 kB/s | 38 kB 00:00 2024-09-10T18:14:39.083 INFO:teuthology.orchestra.run.smithi150.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.5 MB/s | 245 kB 00:00 2024-09-10T18:14:39.134 INFO:teuthology.orchestra.run.smithi150.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 7.3 MB/s | 3.0 MB 00:00 2024-09-10T18:14:39.201 INFO:teuthology.orchestra.run.smithi150.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 13 MB/s | 4.9 MB 00:00 2024-09-10T18:14:39.226 INFO:teuthology.orchestra.run.smithi150.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.7 MB/s | 250 kB 00:00 2024-09-10T18:14:39.233 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9. 6/7 2024-09-10T18:14:39.271 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarc 7/7 2024-09-10T18:14:39.393 INFO:teuthology.orchestra.run.smithi150.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-09-10T18:14:39.585 INFO:teuthology.orchestra.run.smithi150.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 42 MB/s | 16 MB 00:00 2024-09-10T18:14:39.586 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:14:39.586 INFO:teuthology.orchestra.run.smithi150.stdout:Total 27 MB/s | 38 MB 00:01 2024-09-10T18:14:39.942 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T18:14:39.968 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T18:14:39.968 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T18:14:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarc 7/7 2024-09-10T18:14:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9. 1/7 2024-09-10T18:14:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarc 2/7 2024-09-10T18:14:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.n 3/7 2024-09-10T18:14:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-10T18:14:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-10T18:14:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-10T18:14:40.490 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T18:14:40.490 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T18:14:40.634 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-10T18:14:40.634 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:40.634 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T18:14:40.634 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:14:40.634 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:14:40.634 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:14:40.635 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-10T18:14:40.635 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-10T18:14:40.635 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T18:14:40.635 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T18:14:40.635 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:40.635 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T18:14:40.868 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-10T18:14:41.243 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T18:14:41.312 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-10T18:14:41.357 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-10T18:14:41.426 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:01 ago on Tue 10 Sep 2024 06:13:40 PM UTC. 2024-09-10T18:14:41.461 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-10T18:14:41.501 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-10T18:14:41.560 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T18:14:41.561 INFO:teuthology.orchestra.run.smithi115.stdout:======================================================================================= 2024-09-10T18:14:41.561 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-10T18:14:41.561 INFO:teuthology.orchestra.run.smithi115.stdout:======================================================================================= 2024-09-10T18:14:41.561 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T18:14:41.561 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 7.4 M 2024-09-10T18:14:41.561 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-10T18:14:41.562 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T18:14:41.562 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T18:14:41.562 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T18:14:41.562 INFO:teuthology.orchestra.run.smithi115.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T18:14:41.562 INFO:teuthology.orchestra.run.smithi115.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T18:14:41.562 INFO:teuthology.orchestra.run.smithi115.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T18:14:41.562 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T18:14:41.562 INFO:teuthology.orchestra.run.smithi115.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T18:14:41.562 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T18:14:41.562 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T18:14:41.562 INFO:teuthology.orchestra.run.smithi115.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T18:14:41.562 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:41.562 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T18:14:41.562 INFO:teuthology.orchestra.run.smithi115.stdout:======================================================================================= 2024-09-10T18:14:41.563 INFO:teuthology.orchestra.run.smithi115.stdout:Install 12 Packages 2024-09-10T18:14:41.563 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:41.563 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 38 M 2024-09-10T18:14:41.563 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 212 M 2024-09-10T18:14:41.563 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T18:14:41.686 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-10T18:14:41.716 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-10T18:14:41.783 INFO:teuthology.orchestra.run.smithi115.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 191 kB 00:00 2024-09-10T18:14:41.799 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-10T18:14:41.866 INFO:teuthology.orchestra.run.smithi115.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 3.3 MB/s | 801 kB 00:00 2024-09-10T18:14:41.891 INFO:teuthology.orchestra.run.smithi115.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 301 kB/s | 33 kB 00:00 2024-09-10T18:14:41.992 INFO:teuthology.orchestra.run.smithi115.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 182 kB/s | 18 kB 00:00 2024-09-10T18:14:42.050 INFO:teuthology.orchestra.run.smithi115.stdout:(5/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 16 MB/s | 3.0 MB 00:00 2024-09-10T18:14:42.075 INFO:teuthology.orchestra.run.smithi115.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 461 kB/s | 38 kB 00:00 2024-09-10T18:14:42.168 INFO:teuthology.orchestra.run.smithi115.stdout:(7/12): ceph-mgr-diskprediction-local-17.2.7-16 14 MB/s | 7.4 MB 00:00 2024-09-10T18:14:42.193 INFO:teuthology.orchestra.run.smithi115.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 2.0 MB/s | 245 kB 00:00 2024-09-10T18:14:42.276 INFO:teuthology.orchestra.run.smithi115.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 22 MB/s | 4.9 MB 00:00 2024-09-10T18:14:42.301 INFO:teuthology.orchestra.run.smithi115.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.3 MB/s | 250 kB 00:00 2024-09-10T18:14:42.438 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-10T18:14:42.557 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-10T18:14:42.693 INFO:teuthology.orchestra.run.smithi115.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 39 MB/s | 16 MB 00:00 2024-09-10T18:14:42.760 INFO:teuthology.orchestra.run.smithi115.stdout:(12/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.6 MB/s | 5.1 MB 00:00 2024-09-10T18:14:42.765 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:14:42.765 INFO:teuthology.orchestra.run.smithi115.stdout:Total 32 MB/s | 38 MB 00:01 2024-09-10T18:14:43.112 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T18:14:43.139 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T18:14:43.139 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T18:14:43.637 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T18:14:43.637 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T18:14:43.789 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T18:14:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T18:14:44.461 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-10T18:14:44.509 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-10T18:14:44.607 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-10T18:14:44.651 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-10T18:14:44.821 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-10T18:14:44.836 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-10T18:14:44.849 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57 12/12 2024-09-10T18:14:44.867 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-10T18:14:44.946 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-10T18:14:45.584 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-10T18:14:45.696 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-10T18:14:46.914 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T18:14:47.456 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57 12/12 2024-09-10T18:14:47.456 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57 1/12 2024-09-10T18:14:47.456 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-10T18:14:47.456 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-10T18:14:47.457 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-10T18:14:47.457 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-10T18:14:47.457 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-10T18:14:47.457 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-10T18:14:47.457 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-10T18:14:47.457 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-10T18:14:47.457 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T18:14:47.457 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-10T18:14:47.841 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-10T18:14:47.841 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:47.841 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-10T18:14:47.841 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:14:47.841 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T18:14:47.842 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T18:14:47.842 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T18:14:47.842 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T18:14:47.842 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T18:14:47.842 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T18:14:47.842 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T18:14:47.843 INFO:teuthology.orchestra.run.smithi150.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T18:14:47.843 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T18:14:47.843 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T18:14:47.843 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T18:14:47.843 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:47.844 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T18:14:47.959 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-10T18:14:47.991 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57 12/12 2024-09-10T18:14:48.212 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-mgr-rook 2024-09-10T18:14:48.787 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:10 ago on Tue 10 Sep 2024 06:13:38 PM UTC. 2024-09-10T18:14:48.924 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T18:14:48.926 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:14:48.926 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-09-10T18:14:48.926 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:14:48.926 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-10T18:14:48.926 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 47 k 2024-09-10T18:14:48.926 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-09-10T18:14:48.926 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T18:14:48.926 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T18:14:48.926 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T18:14:48.926 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T18:14:48.926 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T18:14:48.926 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T18:14:48.927 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T18:14:48.927 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T18:14:48.927 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T18:14:48.927 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T18:14:48.927 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T18:14:48.927 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T18:14:48.927 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:48.927 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T18:14:48.927 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:14:48.927 INFO:teuthology.orchestra.run.smithi150.stdout:Install 13 Packages 2024-09-10T18:14:48.927 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:48.928 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 2.2 M 2024-09-10T18:14:48.928 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 26 M 2024-09-10T18:14:48.928 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-10T18:14:49.273 INFO:teuthology.orchestra.run.smithi150.stdout:(1/13): ceph-mgr-rook-17.2.7-1606.g5be57395.el9 291 kB/s | 47 kB 00:00 2024-09-10T18:14:49.623 INFO:teuthology.orchestra.run.smithi150.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 37 kB/s | 19 kB 00:00 2024-09-10T18:14:49.665 INFO:teuthology.orchestra.run.smithi150.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 46 kB/s | 26 kB 00:00 2024-09-10T18:14:50.124 INFO:teuthology.orchestra.run.smithi150.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 261 kB/s | 222 kB 00:00 2024-09-10T18:14:50.365 INFO:teuthology.orchestra.run.smithi150.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 214 kB/s | 159 kB 00:00 2024-09-10T18:14:50.415 INFO:teuthology.orchestra.run.smithi150.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 768 kB/s | 32 kB 00:00 2024-09-10T18:14:50.441 INFO:teuthology.orchestra.run.smithi150.stdout:(7/13): python3-certifi-2023.05.07-4.el9.noarch 566 kB/s | 14 kB 00:00 2024-09-10T18:14:50.466 INFO:teuthology.orchestra.run.smithi150.stdout:(8/13): python3-google-auth-2.34.0-1.el9.noarch 8.6 MB/s | 220 kB 00:00 2024-09-10T18:14:50.508 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57 12/12 2024-09-10T18:14:50.508 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57 1/12 2024-09-10T18:14:50.508 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-10T18:14:50.508 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-10T18:14:50.508 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-10T18:14:50.509 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-10T18:14:50.509 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-10T18:14:50.509 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-10T18:14:50.509 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-10T18:14:50.509 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-10T18:14:50.509 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T18:14:50.509 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-10T18:14:50.510 INFO:teuthology.orchestra.run.smithi150.stdout:(9/13): python3-kubernetes-26.1.0-2.el9.noarch. 25 MB/s | 1.0 MB 00:00 2024-09-10T18:14:50.525 INFO:teuthology.orchestra.run.smithi150.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 3.5 MB/s | 59 kB 00:00 2024-09-10T18:14:50.550 INFO:teuthology.orchestra.run.smithi150.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.5 MB/s | 90 kB 00:00 2024-09-10T18:14:50.651 INFO:teuthology.orchestra.run.smithi150.stdout:(12/13): python3-pyasn1-modules-0.4.8-6.el9.noa 283 kB/s | 279 kB 00:00 2024-09-10T18:14:50.759 INFO:teuthology.orchestra.run.smithi150.stdout:(13/13): python3-requests-oauthlib-1.3.0-12.el9 84 kB/s | 54 kB 00:00 2024-09-10T18:14:50.761 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:14:50.761 INFO:teuthology.orchestra.run.smithi150.stdout:Total 1.2 MB/s | 2.2 MB 00:01 2024-09-10T18:14:50.831 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T18:14:50.847 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T18:14:50.847 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T18:14:50.882 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-10T18:14:50.883 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:50.883 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T18:14:50.883 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:14:50.884 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T18:14:50.884 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T18:14:50.884 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T18:14:50.884 INFO:teuthology.orchestra.run.smithi115.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T18:14:50.884 INFO:teuthology.orchestra.run.smithi115.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T18:14:50.884 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T18:14:50.885 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T18:14:50.885 INFO:teuthology.orchestra.run.smithi115.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T18:14:50.885 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T18:14:50.885 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T18:14:50.885 INFO:teuthology.orchestra.run.smithi115.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T18:14:50.886 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:50.886 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T18:14:50.938 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T18:14:50.939 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T18:14:51.276 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr-rook 2024-09-10T18:14:51.439 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T18:14:51.615 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-10T18:14:51.676 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-10T18:14:51.732 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-10T18:14:51.769 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-10T18:14:51.811 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-10T18:14:51.853 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:11 ago on Tue 10 Sep 2024 06:13:40 PM UTC. 2024-09-10T18:14:51.913 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-10T18:14:51.989 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T18:14:51.989 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:14:51.989 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-10T18:14:51.990 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:14:51.990 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T18:14:51.990 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-rook noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 47 k 2024-09-10T18:14:51.990 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-10T18:14:51.990 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T18:14:51.990 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T18:14:51.990 INFO:teuthology.orchestra.run.smithi115.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T18:14:51.990 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T18:14:51.990 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T18:14:51.990 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T18:14:51.990 INFO:teuthology.orchestra.run.smithi115.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T18:14:51.990 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T18:14:51.990 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T18:14:51.991 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T18:14:51.991 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T18:14:51.991 INFO:teuthology.orchestra.run.smithi115.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T18:14:51.991 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:51.991 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T18:14:51.991 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:14:51.991 INFO:teuthology.orchestra.run.smithi115.stdout:Install 13 Packages 2024-09-10T18:14:51.991 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:51.991 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 2.2 M 2024-09-10T18:14:51.991 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 26 M 2024-09-10T18:14:51.991 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T18:14:52.022 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-10T18:14:52.079 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-10T18:14:52.182 INFO:teuthology.orchestra.run.smithi115.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 224 kB/s | 19 kB 00:00 2024-09-10T18:14:52.207 INFO:teuthology.orchestra.run.smithi115.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 231 kB/s | 26 kB 00:00 2024-09-10T18:14:52.274 INFO:teuthology.orchestra.run.smithi115.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.4 MB/s | 222 kB 00:00 2024-09-10T18:14:52.299 INFO:teuthology.orchestra.run.smithi115.stdout:(4/13): ceph-mgr-rook-17.2.7-1606.g5be57395.el9 234 kB/s | 47 kB 00:00 2024-09-10T18:14:52.325 INFO:teuthology.orchestra.run.smithi115.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-10T18:14:52.350 INFO:teuthology.orchestra.run.smithi115.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-09-10T18:14:52.375 INFO:teuthology.orchestra.run.smithi115.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.7 MB/s | 279 kB 00:00 2024-09-10T18:14:52.426 INFO:teuthology.orchestra.run.smithi115.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 320 kB/s | 32 kB 00:00 2024-09-10T18:14:52.451 INFO:teuthology.orchestra.run.smithi115.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 140 kB/s | 14 kB 00:00 2024-09-10T18:14:52.484 INFO:teuthology.orchestra.run.smithi115.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.2 MB/s | 220 kB 00:00 2024-09-10T18:14:52.510 INFO:teuthology.orchestra.run.smithi115.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-09-10T18:14:52.535 INFO:teuthology.orchestra.run.smithi115.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.7 MB/s | 90 kB 00:00 2024-09-10T18:14:52.569 INFO:teuthology.orchestra.run.smithi115.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.3 MB/s | 1.0 MB 00:00 2024-09-10T18:14:52.571 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:14:52.571 INFO:teuthology.orchestra.run.smithi115.stdout:Total 3.8 MB/s | 2.2 MB 00:00 2024-09-10T18:14:52.632 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T18:14:52.646 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T18:14:52.647 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T18:14:52.733 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T18:14:52.735 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T18:14:52.794 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-10T18:14:52.843 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-10T18:14:52.876 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-10T18:14:52.923 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-10T18:14:52.938 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 13/13 2024-09-10T18:14:53.237 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T18:14:53.406 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-10T18:14:53.461 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-10T18:14:53.516 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-10T18:14:53.554 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-10T18:14:53.595 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-10T18:14:53.697 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-10T18:14:53.742 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 13/13 2024-09-10T18:14:53.742 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 1/13 2024-09-10T18:14:53.742 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-10T18:14:53.742 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-10T18:14:53.742 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-10T18:14:53.742 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-10T18:14:53.742 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-10T18:14:53.742 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-10T18:14:53.742 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-10T18:14:53.743 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-10T18:14:53.743 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-10T18:14:53.743 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-10T18:14:53.743 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-10T18:14:53.807 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-10T18:14:53.855 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-10T18:14:54.080 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-10T18:14:54.080 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:54.080 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-10T18:14:54.080 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:14:54.080 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T18:14:54.080 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T18:14:54.080 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T18:14:54.081 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T18:14:54.081 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T18:14:54.081 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T18:14:54.081 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T18:14:54.081 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T18:14:54.081 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T18:14:54.081 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T18:14:54.081 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T18:14:54.081 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T18:14:54.081 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:54.081 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T18:14:54.340 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-mgr-cephadm 2024-09-10T18:14:54.567 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-10T18:14:54.619 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-10T18:14:54.653 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-10T18:14:54.698 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-10T18:14:54.723 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 13/13 2024-09-10T18:14:54.921 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:16 ago on Tue 10 Sep 2024 06:13:38 PM UTC. 2024-09-10T18:14:55.057 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T18:14:55.057 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:14:55.057 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-09-10T18:14:55.057 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:14:55.058 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-10T18:14:55.058 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 124 k 2024-09-10T18:14:55.058 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-09-10T18:14:55.058 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T18:14:55.058 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T18:14:55.058 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T18:14:55.058 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T18:14:55.058 INFO:teuthology.orchestra.run.smithi150.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T18:14:55.058 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:55.058 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T18:14:55.058 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:14:55.058 INFO:teuthology.orchestra.run.smithi150.stdout:Install 6 Packages 2024-09-10T18:14:55.059 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:14:55.059 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 7.0 M 2024-09-10T18:14:55.059 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 33 M 2024-09-10T18:14:55.059 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-10T18:14:55.469 INFO:teuthology.orchestra.run.smithi150.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.1 MB/s | 248 kB 00:00 2024-09-10T18:14:55.494 INFO:teuthology.orchestra.run.smithi150.stdout:(2/6): ceph-mgr-cephadm-17.2.7-1606.g5be57395.e 521 kB/s | 124 kB 00:00 2024-09-10T18:14:55.538 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 13/13 2024-09-10T18:14:55.539 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 1/13 2024-09-10T18:14:55.539 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-10T18:14:55.539 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-10T18:14:55.539 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-10T18:14:55.539 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-10T18:14:55.539 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-10T18:14:55.539 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-10T18:14:55.539 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-10T18:14:55.539 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-10T18:14:55.540 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-10T18:14:55.540 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-10T18:14:55.540 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-10T18:14:55.544 INFO:teuthology.orchestra.run.smithi150.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 7.1 MB/s | 548 kB 00:00 2024-09-10T18:14:55.570 INFO:teuthology.orchestra.run.smithi150.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 767 kB/s | 58 kB 00:00 2024-09-10T18:14:55.595 INFO:teuthology.orchestra.run.smithi150.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.4 MB/s | 74 kB 00:00 2024-09-10T18:14:55.866 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-10T18:14:55.866 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:55.866 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T18:14:55.866 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:14:55.866 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T18:14:55.866 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T18:14:55.867 INFO:teuthology.orchestra.run.smithi115.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T18:14:55.867 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T18:14:55.867 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T18:14:55.867 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T18:14:55.867 INFO:teuthology.orchestra.run.smithi115.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T18:14:55.867 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T18:14:55.867 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T18:14:55.867 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T18:14:55.867 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T18:14:55.867 INFO:teuthology.orchestra.run.smithi115.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T18:14:55.867 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:55.867 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T18:14:56.145 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr-cephadm 2024-09-10T18:14:56.725 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:16 ago on Tue 10 Sep 2024 06:13:40 PM UTC. 2024-09-10T18:14:56.864 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T18:14:56.865 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:14:56.865 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-10T18:14:56.865 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:14:56.865 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T18:14:56.865 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 124 k 2024-09-10T18:14:56.866 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-10T18:14:56.866 INFO:teuthology.orchestra.run.smithi115.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T18:14:56.866 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T18:14:56.866 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T18:14:56.866 INFO:teuthology.orchestra.run.smithi115.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T18:14:56.866 INFO:teuthology.orchestra.run.smithi115.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T18:14:56.866 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:56.866 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T18:14:56.866 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:14:56.866 INFO:teuthology.orchestra.run.smithi115.stdout:Install 6 Packages 2024-09-10T18:14:56.866 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:14:56.867 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 7.0 M 2024-09-10T18:14:56.867 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 33 M 2024-09-10T18:14:56.867 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T18:14:57.146 INFO:teuthology.orchestra.run.smithi115.stdout:(1/6): ceph-mgr-cephadm-17.2.7-1606.g5be57395.e 623 kB/s | 124 kB 00:00 2024-09-10T18:14:57.196 INFO:teuthology.orchestra.run.smithi115.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 11 MB/s | 548 kB 00:00 2024-09-10T18:14:57.221 INFO:teuthology.orchestra.run.smithi115.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.3 MB/s | 58 kB 00:00 2024-09-10T18:14:57.254 INFO:teuthology.orchestra.run.smithi115.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.7 MB/s | 74 kB 00:00 2024-09-10T18:14:57.670 INFO:teuthology.orchestra.run.smithi150.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 2.5 MB/s | 6.0 MB 00:02 2024-09-10T18:14:57.673 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:14:57.674 INFO:teuthology.orchestra.run.smithi150.stdout:Total 2.7 MB/s | 7.0 MB 00:02 2024-09-10T18:14:57.747 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T18:14:57.759 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T18:14:57.759 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T18:14:57.771 INFO:teuthology.orchestra.run.smithi115.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 301 kB/s | 248 kB 00:00 2024-09-10T18:14:57.861 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T18:14:57.861 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T18:14:58.130 INFO:teuthology.orchestra.run.smithi115.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 5.0 MB/s | 6.0 MB 00:01 2024-09-10T18:14:58.132 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:14:58.132 INFO:teuthology.orchestra.run.smithi115.stdout:Total 5.5 MB/s | 7.0 MB 00:01 2024-09-10T18:14:58.168 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T18:14:58.203 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T18:14:58.214 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T18:14:58.214 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T18:14:58.314 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T18:14:58.314 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T18:14:58.336 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-10T18:14:58.405 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-10T18:14:58.618 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T18:14:58.777 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-10T18:14:58.831 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-10T18:14:58.941 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-10T18:14:59.033 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-10T18:14:59.088 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-10T18:14:59.117 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 6/6 2024-09-10T18:14:59.359 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-10T18:14:59.442 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-10T18:14:59.498 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-10T18:14:59.527 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 6/6 2024-09-10T18:14:59.857 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 6/6 2024-09-10T18:14:59.857 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/6 2024-09-10T18:14:59.857 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-10T18:14:59.857 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-10T18:14:59.858 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-10T18:14:59.858 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-10T18:15:00.120 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-10T18:15:00.120 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:15:00.120 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-10T18:15:00.120 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:15:00.120 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T18:15:00.121 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T18:15:00.121 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T18:15:00.121 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T18:15:00.121 INFO:teuthology.orchestra.run.smithi150.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T18:15:00.121 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:15:00.122 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T18:15:00.369 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 6/6 2024-09-10T18:15:00.369 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/6 2024-09-10T18:15:00.369 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-10T18:15:00.369 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-10T18:15:00.370 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-10T18:15:00.370 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-10T18:15:00.376 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-fuse 2024-09-10T18:15:00.647 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-10T18:15:00.647 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:15:00.648 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T18:15:00.648 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:15:00.648 INFO:teuthology.orchestra.run.smithi115.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T18:15:00.648 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T18:15:00.648 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T18:15:00.648 INFO:teuthology.orchestra.run.smithi115.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T18:15:00.648 INFO:teuthology.orchestra.run.smithi115.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T18:15:00.648 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:15:00.648 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T18:15:00.909 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-fuse 2024-09-10T18:15:00.943 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:22 ago on Tue 10 Sep 2024 06:13:38 PM UTC. 2024-09-10T18:15:01.075 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T18:15:01.077 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:15:01.077 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-09-10T18:15:01.077 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:15:01.077 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-10T18:15:01.077 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 813 k 2024-09-10T18:15:01.077 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-09-10T18:15:01.077 INFO:teuthology.orchestra.run.smithi150.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T18:15:01.077 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:15:01.077 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T18:15:01.077 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:15:01.078 INFO:teuthology.orchestra.run.smithi150.stdout:Install 2 Packages 2024-09-10T18:15:01.078 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:15:01.078 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 893 k 2024-09-10T18:15:01.078 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 2.6 M 2024-09-10T18:15:01.078 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-10T18:15:01.464 INFO:teuthology.orchestra.run.smithi150.stdout:(1/2): ceph-fuse-17.2.7-1606.g5be57395.el9.x86_ 3.0 MB/s | 813 kB 00:00 2024-09-10T18:15:01.472 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:21 ago on Tue 10 Sep 2024 06:13:40 PM UTC. 2024-09-10T18:15:01.606 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T18:15:01.609 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:15:01.609 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-10T18:15:01.609 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:15:01.609 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T18:15:01.609 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-fuse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 813 k 2024-09-10T18:15:01.609 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-10T18:15:01.609 INFO:teuthology.orchestra.run.smithi115.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T18:15:01.609 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:15:01.610 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T18:15:01.610 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:15:01.610 INFO:teuthology.orchestra.run.smithi115.stdout:Install 2 Packages 2024-09-10T18:15:01.610 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:15:01.610 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 893 k 2024-09-10T18:15:01.610 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 2.6 M 2024-09-10T18:15:01.610 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T18:15:01.655 INFO:teuthology.orchestra.run.smithi150.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 174 kB/s | 80 kB 00:00 2024-09-10T18:15:01.656 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:15:01.656 INFO:teuthology.orchestra.run.smithi150.stdout:Total 1.5 MB/s | 893 kB 00:00 2024-09-10T18:15:01.668 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T18:15:01.689 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T18:15:01.689 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T18:15:01.767 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T18:15:01.767 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T18:15:01.953 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T18:15:02.073 INFO:teuthology.orchestra.run.smithi115.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 430 kB/s | 80 kB 00:00 2024-09-10T18:15:02.075 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-10T18:15:02.145 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2/2 2024-09-10T18:15:02.165 INFO:teuthology.orchestra.run.smithi115.stdout:(2/2): ceph-fuse-17.2.7-1606.g5be57395.el9.x86_ 2.9 MB/s | 813 kB 00:00 2024-09-10T18:15:02.165 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:15:02.166 INFO:teuthology.orchestra.run.smithi115.stdout:Total 1.6 MB/s | 893 kB 00:00 2024-09-10T18:15:02.178 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T18:15:02.199 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T18:15:02.199 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T18:15:02.280 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T18:15:02.281 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T18:15:02.462 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T18:15:02.586 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-10T18:15:02.646 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2/2 2024-09-10T18:15:02.805 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2/2 2024-09-10T18:15:02.806 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/2 2024-09-10T18:15:03.077 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-10T18:15:03.077 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:15:03.077 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-10T18:15:03.077 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-10T18:15:03.078 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:15:03.078 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T18:15:03.367 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-volume 2024-09-10T18:15:03.378 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2/2 2024-09-10T18:15:03.378 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/2 2024-09-10T18:15:03.641 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-10T18:15:03.642 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:15:03.642 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T18:15:03.642 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-10T18:15:03.642 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:15:03.642 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T18:15:03.935 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-volume 2024-09-10T18:15:03.943 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:25 ago on Tue 10 Sep 2024 06:13:38 PM UTC. 2024-09-10T18:15:04.084 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T18:15:04.085 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:15:04.085 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-09-10T18:15:04.085 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:15:04.085 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-10T18:15:04.085 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 270 k 2024-09-10T18:15:04.085 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:15:04.085 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T18:15:04.085 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:15:04.085 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-09-10T18:15:04.085 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:15:04.085 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 270 k 2024-09-10T18:15:04.085 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 1.3 M 2024-09-10T18:15:04.086 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-10T18:15:04.326 INFO:teuthology.orchestra.run.smithi150.stdout:ceph-volume-17.2.7-1606.g5be57395.el9.noarch.rp 1.1 MB/s | 270 kB 00:00 2024-09-10T18:15:04.327 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:15:04.327 INFO:teuthology.orchestra.run.smithi150.stdout:Total 1.1 MB/s | 270 kB 00:00 2024-09-10T18:15:04.327 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T18:15:04.335 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T18:15:04.335 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T18:15:04.361 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T18:15:04.362 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T18:15:04.506 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:24 ago on Tue 10 Sep 2024 06:13:40 PM UTC. 2024-09-10T18:15:04.572 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T18:15:04.636 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:15:04.642 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T18:15:04.643 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:15:04.643 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-10T18:15:04.643 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:15:04.643 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T18:15:04.643 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-volume noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 270 k 2024-09-10T18:15:04.643 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:15:04.644 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T18:15:04.644 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:15:04.644 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-10T18:15:04.644 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:15:04.644 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 270 k 2024-09-10T18:15:04.644 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 1.3 M 2024-09-10T18:15:04.644 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T18:15:04.652 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:15:04.652 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:15:04.652 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T18:15:04.653 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:15:04.870 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-volume-17.2.7-1606.g5be57395.el9.noarch.rp 1.2 MB/s | 270 kB 00:00 2024-09-10T18:15:04.870 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:15:04.870 INFO:teuthology.orchestra.run.smithi115.stdout:Total 1.2 MB/s | 270 kB 00:00 2024-09-10T18:15:04.871 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T18:15:04.879 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T18:15:04.879 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T18:15:04.908 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T18:15:04.909 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T18:15:05.112 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T18:15:05.172 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:15:05.188 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:15:05.188 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:15:05.188 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T18:15:05.188 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:15:05.648 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:15:05.648 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:15:05.648 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-10T18:15:05.648 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:15:05.648 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:15:05.648 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T18:15:05.956 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install librados-devel 2024-09-10T18:15:06.158 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:15:06.158 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:15:06.158 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T18:15:06.158 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:15:06.158 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:15:06.158 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T18:15:06.464 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install librados-devel 2024-09-10T18:15:06.542 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:28 ago on Tue 10 Sep 2024 06:13:38 PM UTC. 2024-09-10T18:15:06.681 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T18:15:06.682 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:15:06.682 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repo Size 2024-09-10T18:15:06.682 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:15:06.682 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-10T18:15:06.682 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel x86_64 2:17.2.7-1606.g5be57395.el9 ceph 126 k 2024-09-10T18:15:06.683 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:15:06.683 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T18:15:06.683 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:15:06.683 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-09-10T18:15:06.683 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:15:06.683 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 126 k 2024-09-10T18:15:06.683 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 448 k 2024-09-10T18:15:06.683 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-10T18:15:06.889 INFO:teuthology.orchestra.run.smithi150.stdout:librados-devel-17.2.7-1606.g5be57395.el9.x86_64 612 kB/s | 126 kB 00:00 2024-09-10T18:15:06.890 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:15:06.890 INFO:teuthology.orchestra.run.smithi150.stdout:Total 607 kB/s | 126 kB 00:00 2024-09-10T18:15:06.890 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T18:15:06.896 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T18:15:06.896 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T18:15:06.961 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T18:15:06.962 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T18:15:07.040 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:27 ago on Tue 10 Sep 2024 06:13:40 PM UTC. 2024-09-10T18:15:07.126 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T18:15:07.174 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T18:15:07.175 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:15:07.175 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repo Size 2024-09-10T18:15:07.175 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:15:07.175 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T18:15:07.175 INFO:teuthology.orchestra.run.smithi115.stdout: librados-devel x86_64 2:17.2.7-1606.g5be57395.el9 ceph 126 k 2024-09-10T18:15:07.175 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:15:07.175 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T18:15:07.175 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:15:07.175 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-10T18:15:07.175 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:15:07.176 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 126 k 2024-09-10T18:15:07.176 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 448 k 2024-09-10T18:15:07.176 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T18:15:07.220 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:15:07.383 INFO:teuthology.orchestra.run.smithi115.stdout:librados-devel-17.2.7-1606.g5be57395.el9.x86_64 610 kB/s | 126 kB 00:00 2024-09-10T18:15:07.384 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:15:07.384 INFO:teuthology.orchestra.run.smithi115.stdout:Total 604 kB/s | 126 kB 00:00 2024-09-10T18:15:07.384 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T18:15:07.390 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T18:15:07.390 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T18:15:07.454 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T18:15:07.454 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T18:15:07.613 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:15:07.614 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T18:15:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:15:07.881 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:15:07.882 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:15:07.882 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-10T18:15:07.882 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:15:07.882 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:15:07.882 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T18:15:08.132 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:15:08.179 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install libcephfs2 2024-09-10T18:15:08.387 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:15:08.387 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:15:08.387 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T18:15:08.387 INFO:teuthology.orchestra.run.smithi115.stdout: librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:15:08.387 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:15:08.387 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T18:15:08.678 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install libcephfs2 2024-09-10T18:15:08.758 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:30 ago on Tue 10 Sep 2024 06:13:38 PM UTC. 2024-09-10T18:15:08.850 INFO:teuthology.orchestra.run.smithi150.stdout:Package libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:15:08.895 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T18:15:08.896 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-09-10T18:15:08.896 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T18:15:08.964 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install libcephfs-devel 2024-09-10T18:15:09.255 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:29 ago on Tue 10 Sep 2024 06:13:40 PM UTC. 2024-09-10T18:15:09.348 INFO:teuthology.orchestra.run.smithi115.stdout:Package libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:15:09.393 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T18:15:09.394 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-10T18:15:09.394 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T18:15:09.479 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install libcephfs-devel 2024-09-10T18:15:09.533 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:31 ago on Tue 10 Sep 2024 06:13:38 PM UTC. 2024-09-10T18:15:09.665 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T18:15:09.666 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:15:09.666 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repo Size 2024-09-10T18:15:09.666 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:15:09.666 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-10T18:15:09.666 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel x86_64 2:17.2.7-1606.g5be57395.el9 ceph 25 k 2024-09-10T18:15:09.666 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:15:09.667 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T18:15:09.667 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:15:09.667 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-09-10T18:15:09.667 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:15:09.667 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 25 k 2024-09-10T18:15:09.667 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 110 k 2024-09-10T18:15:09.667 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-10T18:15:09.835 INFO:teuthology.orchestra.run.smithi150.stdout:libcephfs-devel-17.2.7-1606.g5be57395.el9.x86_6 148 kB/s | 25 kB 00:00 2024-09-10T18:15:09.836 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:15:09.836 INFO:teuthology.orchestra.run.smithi150.stdout:Total 147 kB/s | 25 kB 00:00 2024-09-10T18:15:09.836 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T18:15:09.839 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T18:15:09.840 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T18:15:09.841 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T18:15:09.841 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T18:15:09.931 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T18:15:10.041 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:15:10.042 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:30 ago on Tue 10 Sep 2024 06:13:40 PM UTC. 2024-09-10T18:15:10.174 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T18:15:10.175 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:15:10.175 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repo Size 2024-09-10T18:15:10.175 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:15:10.175 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T18:15:10.175 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-devel x86_64 2:17.2.7-1606.g5be57395.el9 ceph 25 k 2024-09-10T18:15:10.175 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:15:10.175 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T18:15:10.175 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:15:10.175 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-10T18:15:10.176 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:15:10.176 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 25 k 2024-09-10T18:15:10.176 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 110 k 2024-09-10T18:15:10.176 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T18:15:10.320 INFO:teuthology.orchestra.run.smithi115.stdout:libcephfs-devel-17.2.7-1606.g5be57395.el9.x86_6 174 kB/s | 25 kB 00:00 2024-09-10T18:15:10.320 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:15:10.320 INFO:teuthology.orchestra.run.smithi115.stdout:Total 172 kB/s | 25 kB 00:00 2024-09-10T18:15:10.320 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T18:15:10.324 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T18:15:10.324 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T18:15:10.325 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T18:15:10.326 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T18:15:10.415 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T18:15:10.424 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:15:10.516 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:15:10.667 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:15:10.667 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:15:10.667 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-10T18:15:10.667 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:15:10.667 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:15:10.667 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T18:15:10.842 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install librados2 2024-09-10T18:15:10.859 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:15:11.086 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:15:11.086 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:15:11.086 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T18:15:11.086 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:15:11.086 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:15:11.086 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T18:15:11.309 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install librados2 2024-09-10T18:15:11.407 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:33 ago on Tue 10 Sep 2024 06:13:38 PM UTC. 2024-09-10T18:15:11.498 INFO:teuthology.orchestra.run.smithi150.stdout:Package librados2-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:15:11.541 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T18:15:11.542 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-09-10T18:15:11.542 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T18:15:11.620 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install librbd1 2024-09-10T18:15:11.872 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:31 ago on Tue 10 Sep 2024 06:13:40 PM UTC. 2024-09-10T18:15:11.962 INFO:teuthology.orchestra.run.smithi115.stdout:Package librados2-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:15:12.005 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T18:15:12.006 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-10T18:15:12.006 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T18:15:12.086 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install librbd1 2024-09-10T18:15:12.189 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:34 ago on Tue 10 Sep 2024 06:13:38 PM UTC. 2024-09-10T18:15:12.279 INFO:teuthology.orchestra.run.smithi150.stdout:Package librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:15:12.322 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T18:15:12.323 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-09-10T18:15:12.323 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T18:15:12.397 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install python3-rados 2024-09-10T18:15:12.652 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:32 ago on Tue 10 Sep 2024 06:13:40 PM UTC. 2024-09-10T18:15:12.742 INFO:teuthology.orchestra.run.smithi115.stdout:Package librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:15:12.785 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T18:15:12.786 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-10T18:15:12.786 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T18:15:12.870 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install python3-rados 2024-09-10T18:15:12.966 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:34 ago on Tue 10 Sep 2024 06:13:38 PM UTC. 2024-09-10T18:15:13.057 INFO:teuthology.orchestra.run.smithi150.stdout:Package python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:15:13.099 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T18:15:13.100 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-09-10T18:15:13.100 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T18:15:13.170 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install python3-rgw 2024-09-10T18:15:13.436 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:33 ago on Tue 10 Sep 2024 06:13:40 PM UTC. 2024-09-10T18:15:13.527 INFO:teuthology.orchestra.run.smithi115.stdout:Package python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:15:13.569 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T18:15:13.570 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-10T18:15:13.570 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T18:15:13.650 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install python3-rgw 2024-09-10T18:15:13.740 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:35 ago on Tue 10 Sep 2024 06:13:38 PM UTC. 2024-09-10T18:15:13.830 INFO:teuthology.orchestra.run.smithi150.stdout:Package python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:15:13.873 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T18:15:13.874 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-09-10T18:15:13.874 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T18:15:13.956 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install python3-cephfs 2024-09-10T18:15:14.215 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:34 ago on Tue 10 Sep 2024 06:13:40 PM UTC. 2024-09-10T18:15:14.305 INFO:teuthology.orchestra.run.smithi115.stdout:Package python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:15:14.349 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T18:15:14.350 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-10T18:15:14.350 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T18:15:14.423 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install python3-cephfs 2024-09-10T18:15:14.525 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:36 ago on Tue 10 Sep 2024 06:13:38 PM UTC. 2024-09-10T18:15:14.617 INFO:teuthology.orchestra.run.smithi150.stdout:Package python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:15:14.661 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T18:15:14.661 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-09-10T18:15:14.662 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T18:15:14.740 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install python3-rbd 2024-09-10T18:15:14.993 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:34 ago on Tue 10 Sep 2024 06:13:40 PM UTC. 2024-09-10T18:15:15.084 INFO:teuthology.orchestra.run.smithi115.stdout:Package python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:15:15.126 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T18:15:15.127 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-10T18:15:15.127 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T18:15:15.215 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install python3-rbd 2024-09-10T18:15:15.306 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:37 ago on Tue 10 Sep 2024 06:13:38 PM UTC. 2024-09-10T18:15:15.396 INFO:teuthology.orchestra.run.smithi150.stdout:Package python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:15:15.439 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T18:15:15.440 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-09-10T18:15:15.440 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T18:15:15.520 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install rbd-fuse 2024-09-10T18:15:15.781 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:35 ago on Tue 10 Sep 2024 06:13:40 PM UTC. 2024-09-10T18:15:15.871 INFO:teuthology.orchestra.run.smithi115.stdout:Package python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:15:15.913 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T18:15:15.914 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-10T18:15:15.914 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T18:15:16.010 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install rbd-fuse 2024-09-10T18:15:16.090 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:38 ago on Tue 10 Sep 2024 06:13:38 PM UTC. 2024-09-10T18:15:16.224 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T18:15:16.225 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:15:16.225 INFO:teuthology.orchestra.run.smithi150.stdout: Package Architecture Version Repository Size 2024-09-10T18:15:16.225 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:15:16.225 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-10T18:15:16.225 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 82 k 2024-09-10T18:15:16.225 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:15:16.225 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T18:15:16.225 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:15:16.226 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-09-10T18:15:16.226 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:15:16.226 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 82 k 2024-09-10T18:15:16.226 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 222 k 2024-09-10T18:15:16.226 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-10T18:15:16.405 INFO:teuthology.orchestra.run.smithi150.stdout:rbd-fuse-17.2.7-1606.g5be57395.el9.x86_64.rpm 458 kB/s | 82 kB 00:00 2024-09-10T18:15:16.405 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:15:16.405 INFO:teuthology.orchestra.run.smithi150.stdout:Total 454 kB/s | 82 kB 00:00 2024-09-10T18:15:16.405 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T18:15:16.412 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T18:15:16.412 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T18:15:16.475 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T18:15:16.475 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T18:15:16.570 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:36 ago on Tue 10 Sep 2024 06:13:40 PM UTC. 2024-09-10T18:15:16.621 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T18:15:16.704 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T18:15:16.705 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:15:16.705 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2024-09-10T18:15:16.705 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:15:16.705 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T18:15:16.705 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-fuse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 82 k 2024-09-10T18:15:16.705 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:15:16.705 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T18:15:16.705 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:15:16.706 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-10T18:15:16.706 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:15:16.706 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 82 k 2024-09-10T18:15:16.706 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 222 k 2024-09-10T18:15:16.706 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T18:15:16.718 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:15:16.881 INFO:teuthology.orchestra.run.smithi115.stdout:rbd-fuse-17.2.7-1606.g5be57395.el9.x86_64.rpm 467 kB/s | 82 kB 00:00 2024-09-10T18:15:16.881 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:15:16.882 INFO:teuthology.orchestra.run.smithi115.stdout:Total 464 kB/s | 82 kB 00:00 2024-09-10T18:15:16.882 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T18:15:16.888 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T18:15:16.888 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T18:15:16.951 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T18:15:16.951 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T18:15:17.089 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:15:17.094 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T18:15:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:15:17.335 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:15:17.335 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:15:17.335 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-10T18:15:17.335 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:15:17.335 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:15:17.335 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T18:15:17.547 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install rbd-mirror 2024-09-10T18:15:17.590 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:15:17.844 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:15:17.844 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:15:17.844 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T18:15:17.844 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:15:17.844 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:15:17.844 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T18:15:18.118 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:40 ago on Tue 10 Sep 2024 06:13:38 PM UTC. 2024-09-10T18:15:18.125 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install rbd-mirror 2024-09-10T18:15:18.255 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T18:15:18.256 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:15:18.256 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repo Size 2024-09-10T18:15:18.256 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:15:18.256 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-10T18:15:18.256 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.9 M 2024-09-10T18:15:18.256 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:15:18.256 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T18:15:18.256 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:15:18.256 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-09-10T18:15:18.257 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:15:18.257 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 2.9 M 2024-09-10T18:15:18.257 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 12 M 2024-09-10T18:15:18.257 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-10T18:15:18.599 INFO:teuthology.orchestra.run.smithi150.stdout:rbd-mirror-17.2.7-1606.g5be57395.el9.x86_64.rpm 8.6 MB/s | 2.9 MB 00:00 2024-09-10T18:15:18.599 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:15:18.600 INFO:teuthology.orchestra.run.smithi150.stdout:Total 8.6 MB/s | 2.9 MB 00:00 2024-09-10T18:15:18.600 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T18:15:18.609 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T18:15:18.609 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T18:15:18.694 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T18:15:18.694 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T18:15:18.695 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:38 ago on Tue 10 Sep 2024 06:13:40 PM UTC. 2024-09-10T18:15:18.831 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T18:15:18.831 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:15:18.831 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repo Size 2024-09-10T18:15:18.832 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:15:18.832 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T18:15:18.832 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-mirror x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.9 M 2024-09-10T18:15:18.832 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:15:18.832 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T18:15:18.832 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:15:18.832 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-10T18:15:18.832 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:15:18.833 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 2.9 M 2024-09-10T18:15:18.833 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 12 M 2024-09-10T18:15:18.833 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T18:15:19.107 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T18:15:19.166 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:15:19.167 INFO:teuthology.orchestra.run.smithi115.stdout:rbd-mirror-17.2.7-1606.g5be57395.el9.x86_64.rpm 8.8 MB/s | 2.9 MB 00:00 2024-09-10T18:15:19.168 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:15:19.169 INFO:teuthology.orchestra.run.smithi115.stdout:Total 8.7 MB/s | 2.9 MB 00:00 2024-09-10T18:15:19.169 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T18:15:19.178 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T18:15:19.178 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T18:15:19.190 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:15:19.191 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:15:19.191 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T18:15:19.191 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T18:15:19.191 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T18:15:19.191 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:15:19.257 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T18:15:19.258 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T18:15:19.669 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T18:15:19.727 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:15:19.751 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:15:19.752 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:15:19.752 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T18:15:19.752 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T18:15:19.752 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T18:15:19.752 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:15:20.282 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:15:20.283 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:15:20.283 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-10T18:15:20.283 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:15:20.283 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:15:20.283 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T18:15:20.493 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install rbd-nbd 2024-09-10T18:15:20.835 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:15:20.835 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:15:20.835 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T18:15:20.836 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:15:20.836 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:15:20.836 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T18:15:21.053 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install rbd-nbd 2024-09-10T18:15:21.081 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:43 ago on Tue 10 Sep 2024 06:13:38 PM UTC. 2024-09-10T18:15:21.216 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T18:15:21.217 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:15:21.217 INFO:teuthology.orchestra.run.smithi150.stdout: Package Architecture Version Repository Size 2024-09-10T18:15:21.217 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:15:21.217 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-10T18:15:21.217 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 165 k 2024-09-10T18:15:21.217 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:15:21.218 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T18:15:21.218 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T18:15:21.218 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-09-10T18:15:21.218 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:15:21.218 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 165 k 2024-09-10T18:15:21.218 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 477 k 2024-09-10T18:15:21.218 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-10T18:15:21.418 INFO:teuthology.orchestra.run.smithi150.stdout:rbd-nbd-17.2.7-1606.g5be57395.el9.x86_64.rpm 825 kB/s | 165 kB 00:00 2024-09-10T18:15:21.419 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:15:21.419 INFO:teuthology.orchestra.run.smithi150.stdout:Total 819 kB/s | 165 kB 00:00 2024-09-10T18:15:21.419 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T18:15:21.427 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T18:15:21.427 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T18:15:21.492 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T18:15:21.492 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T18:15:21.629 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:41 ago on Tue 10 Sep 2024 06:13:40 PM UTC. 2024-09-10T18:15:21.647 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T18:15:21.758 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:15:21.764 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T18:15:21.764 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:15:21.764 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2024-09-10T18:15:21.765 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:15:21.765 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T18:15:21.765 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-nbd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 165 k 2024-09-10T18:15:21.765 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:15:21.765 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T18:15:21.765 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T18:15:21.765 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-10T18:15:21.765 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:15:21.765 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 165 k 2024-09-10T18:15:21.766 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 477 k 2024-09-10T18:15:21.766 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T18:15:21.988 INFO:teuthology.orchestra.run.smithi115.stdout:rbd-nbd-17.2.7-1606.g5be57395.el9.x86_64.rpm 741 kB/s | 165 kB 00:00 2024-09-10T18:15:21.988 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:15:21.989 INFO:teuthology.orchestra.run.smithi115.stdout:Total 737 kB/s | 165 kB 00:00 2024-09-10T18:15:21.989 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T18:15:21.996 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T18:15:21.996 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T18:15:22.060 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T18:15:22.060 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T18:15:22.183 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:15:22.209 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T18:15:22.302 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:15:22.450 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:15:22.450 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:15:22.450 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-10T18:15:22.451 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:15:22.451 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:15:22.451 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T18:15:22.654 DEBUG:teuthology.parallel:result is None 2024-09-10T18:15:22.728 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:15:22.976 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:15:22.976 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:15:22.977 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T18:15:22.977 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:15:22.977 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:15:22.977 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T18:15:23.210 DEBUG:teuthology.parallel:result is None 2024-09-10T18:15:23.210 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:15:23.351 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T18:15:23.372 INFO:teuthology.orchestra.run.smithi115.stdout:17.2.7-1606.g5be57395.el9 2024-09-10T18:15:23.373 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1606.g5be57395.el9 2024-09-10T18:15:23.373 INFO:teuthology.task.install:The correct ceph version 17.2.7-1606.g5be57395 is installed. 2024-09-10T18:15:23.375 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:15:23.510 DEBUG:teuthology.orchestra.run.smithi150:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T18:15:23.531 INFO:teuthology.orchestra.run.smithi150.stdout:17.2.7-1606.g5be57395.el9 2024-09-10T18:15:23.532 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1606.g5be57395.el9 2024-09-10T18:15:23.532 INFO:teuthology.task.install:The correct ceph version 17.2.7-1606.g5be57395 is installed. 2024-09-10T18:15:23.534 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T18:15:23.534 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T18:15:23.534 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T18:15:23.559 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-10T18:15:23.559 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T18:15:23.599 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T18:15:23.599 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T18:15:23.599 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T18:15:23.624 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T18:15:23.689 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-10T18:15:23.689 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T18:15:23.714 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T18:15:23.779 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T18:15:23.780 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T18:15:23.780 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T18:15:23.804 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T18:15:23.869 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-10T18:15:23.869 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T18:15:23.893 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T18:15:23.958 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T18:15:23.958 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T18:15:23.959 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T18:15:23.983 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T18:15:24.048 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-10T18:15:24.048 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T18:15:24.072 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T18:15:24.136 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-10T18:15:24.229 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'global': {'mon election default strategy': 1, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588', 'cephadm_mode': 'cephadm-package'} 2024-09-10T18:15:24.229 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:15:24.230 INFO:tasks.cephadm:Cluster fsid is a600e180-6fa0-11ef-bcea-c7b262605968 2024-09-10T18:15:24.230 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-10T18:15:24.230 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.115', 'mon.c': '[v2:172.21.15.115:3301,v1:172.21.15.115:6790]', 'mon.b': '172.21.15.150'} 2024-09-10T18:15:24.230 INFO:tasks.cephadm:First mon is mon.a on smithi115 2024-09-10T18:15:24.230 INFO:tasks.cephadm:First mgr is y 2024-09-10T18:15:24.230 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-10T18:15:24.231 DEBUG:teuthology.orchestra.run.smithi115:> sudo hostname $(hostname -s) 2024-09-10T18:15:24.256 DEBUG:teuthology.orchestra.run.smithi150:> sudo hostname $(hostname -s) 2024-09-10T18:15:24.293 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 on all hosts... 2024-09-10T18:15:24.294 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 pull 2024-09-10T18:15:24.298 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 pull 2024-09-10T18:15:25.844 INFO:teuthology.orchestra.run.smithi115.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588... 2024-09-10T18:15:25.891 INFO:teuthology.orchestra.run.smithi150.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588... 2024-09-10T18:16:16.668 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-10T18:16:16.668 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_version": "ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable)", 2024-09-10T18:16:16.668 INFO:teuthology.orchestra.run.smithi150.stdout: "image_id": "4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9", 2024-09-10T18:16:16.668 INFO:teuthology.orchestra.run.smithi150.stdout: "repo_digests": [ 2024-09-10T18:16:16.669 INFO:teuthology.orchestra.run.smithi150.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16" 2024-09-10T18:16:16.669 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-10T18:16:16.669 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-10T18:16:17.367 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-10T18:16:17.367 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_version": "ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable)", 2024-09-10T18:16:17.367 INFO:teuthology.orchestra.run.smithi115.stdout: "image_id": "4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9", 2024-09-10T18:16:17.367 INFO:teuthology.orchestra.run.smithi115.stdout: "repo_digests": [ 2024-09-10T18:16:17.368 INFO:teuthology.orchestra.run.smithi115.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16" 2024-09-10T18:16:17.368 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-10T18:16:17.368 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-10T18:16:17.386 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /etc/ceph 2024-09-10T18:16:17.424 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /etc/ceph 2024-09-10T18:16:17.452 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 777 /etc/ceph 2024-09-10T18:16:17.488 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 777 /etc/ceph 2024-09-10T18:16:17.517 INFO:tasks.cephadm:Writing seed config... 2024-09-10T18:16:17.518 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-10T18:16:17.519 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-10T18:16:17.519 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-09-10T18:16:17.519 INFO:tasks.cephadm: override: [global] ms bind msgr1 = False 2024-09-10T18:16:17.519 INFO:tasks.cephadm: override: [global] ms bind msgr2 = True 2024-09-10T18:16:17.519 INFO:tasks.cephadm: override: [global] ms type = async 2024-09-10T18:16:17.519 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-10T18:16:17.519 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-10T18:16:17.519 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-10T18:16:17.519 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-10T18:16:17.519 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-10T18:16:17.520 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-10T18:16:17.520 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T18:16:17.520 DEBUG:teuthology.orchestra.run.smithi115:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-10T18:16:17.543 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = a600e180-6fa0-11ef-bcea-c7b262605968 mon election default strategy = 1 ms bind msgr1 = False ms bind msgr2 = True ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-09-10T18:16:17.544 DEBUG:teuthology.orchestra.run.smithi115:mon.a> sudo journalctl -f -n 0 -u ceph-a600e180-6fa0-11ef-bcea-c7b262605968@mon.a.service 2024-09-10T18:16:17.586 DEBUG:teuthology.orchestra.run.smithi115:mgr.y> sudo journalctl -f -n 0 -u ceph-a600e180-6fa0-11ef-bcea-c7b262605968@mgr.y.service 2024-09-10T18:16:17.629 INFO:tasks.cephadm:Bootstrapping... 2024-09-10T18:16:17.630 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 -v bootstrap --fsid a600e180-6fa0-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.115 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:16:17.828 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:16:17.828 INFO:teuthology.orchestra.run.smithi115.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588', '-v', 'bootstrap', '--fsid', 'a600e180-6fa0-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.115', '--skip-admin-label'] 2024-09-10T18:16:17.852 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stdout 5.2.2 2024-09-10T18:16:17.852 INFO:teuthology.orchestra.run.smithi115.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-10T18:16:17.852 INFO:teuthology.orchestra.run.smithi115.stdout:Verifying podman|docker is present... 2024-09-10T18:16:17.874 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stdout 5.2.2 2024-09-10T18:16:17.874 INFO:teuthology.orchestra.run.smithi115.stdout:Verifying lvm2 is present... 2024-09-10T18:16:17.875 INFO:teuthology.orchestra.run.smithi115.stdout:Verifying time synchronization is in place... 2024-09-10T18:16:17.882 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T18:16:17.883 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T18:16:17.889 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T18:16:17.889 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout inactive 2024-09-10T18:16:17.896 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout enabled 2024-09-10T18:16:17.902 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout active 2024-09-10T18:16:17.903 INFO:teuthology.orchestra.run.smithi115.stdout:Unit chronyd.service is enabled and running 2024-09-10T18:16:17.903 INFO:teuthology.orchestra.run.smithi115.stdout:Repeating the final host check... 2024-09-10T18:16:17.925 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stdout 5.2.2 2024-09-10T18:16:17.925 INFO:teuthology.orchestra.run.smithi115.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-10T18:16:17.925 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl is present 2024-09-10T18:16:17.925 INFO:teuthology.orchestra.run.smithi115.stdout:lvcreate is present 2024-09-10T18:16:17.932 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T18:16:17.932 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T18:16:17.938 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T18:16:17.938 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout inactive 2024-09-10T18:16:17.945 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout enabled 2024-09-10T18:16:17.952 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout active 2024-09-10T18:16:17.952 INFO:teuthology.orchestra.run.smithi115.stdout:Unit chronyd.service is enabled and running 2024-09-10T18:16:17.952 INFO:teuthology.orchestra.run.smithi115.stdout:Host looks OK 2024-09-10T18:16:17.953 INFO:teuthology.orchestra.run.smithi115.stdout:Cluster fsid: a600e180-6fa0-11ef-bcea-c7b262605968 2024-09-10T18:16:17.953 INFO:teuthology.orchestra.run.smithi115.stdout:Acquiring lock 139936362705680 on /run/cephadm/a600e180-6fa0-11ef-bcea-c7b262605968.lock 2024-09-10T18:16:17.953 INFO:teuthology.orchestra.run.smithi115.stdout:Lock 139936362705680 acquired on /run/cephadm/a600e180-6fa0-11ef-bcea-c7b262605968.lock 2024-09-10T18:16:17.953 INFO:teuthology.orchestra.run.smithi115.stdout:Verifying IP 172.21.15.115 port 3300 ... 2024-09-10T18:16:17.953 INFO:teuthology.orchestra.run.smithi115.stdout:Verifying IP 172.21.15.115 port 6789 ... 2024-09-10T18:16:17.953 INFO:teuthology.orchestra.run.smithi115.stdout:Base mon IP(s) is [172.21.15.115:3300, 172.21.15.115:6789], mon addrv is [v2:172.21.15.115:3300,v1:172.21.15.115:6789] 2024-09-10T18:16:17.957 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.115 metric 100 2024-09-10T18:16:17.957 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.115 metric 100 2024-09-10T18:16:17.960 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-10T18:16:17.960 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-10T18:16:17.960 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-10T18:16:17.962 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-10T18:16:17.963 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-10T18:16:17.963 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T18:16:17.963 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-10T18:16:17.963 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:ce8f/64 scope link noprefixroute 2024-09-10T18:16:17.963 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T18:16:17.963 INFO:teuthology.orchestra.run.smithi115.stdout:Mon IP `172.21.15.115` is in CIDR network `172.21.0.0/20` 2024-09-10T18:16:17.963 INFO:teuthology.orchestra.run.smithi115.stdout:Mon IP `172.21.15.115` is in CIDR network `172.21.0.0/20` 2024-09-10T18:16:17.964 INFO:teuthology.orchestra.run.smithi115.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-10T18:16:17.964 INFO:teuthology.orchestra.run.smithi115.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-10T18:16:17.964 INFO:teuthology.orchestra.run.smithi115.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588... 2024-09-10T18:16:18.401 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stdout 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 2024-09-10T18:16:18.401 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588... 2024-09-10T18:16:18.401 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stderr Getting image source signatures 2024-09-10T18:16:18.401 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stderr Copying blob sha256:5df13064c03763eb21a89b455ab82ba4ec435f51a3f340cae76514aaf2996d4c 2024-09-10T18:16:18.401 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-10T18:16:18.401 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stderr Copying config sha256:4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 2024-09-10T18:16:18.401 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-10T18:16:19.330 INFO:teuthology.orchestra.run.smithi115.stdout:ceph: stdout ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable) 2024-09-10T18:16:19.330 INFO:teuthology.orchestra.run.smithi115.stdout:Ceph version: ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable) 2024-09-10T18:16:19.330 INFO:teuthology.orchestra.run.smithi115.stdout:Extracting ceph user uid/gid from container image... 2024-09-10T18:16:20.393 INFO:teuthology.orchestra.run.smithi115.stdout:stat: stdout 167 167 2024-09-10T18:16:20.393 INFO:teuthology.orchestra.run.smithi115.stdout:Creating initial keys... 2024-09-10T18:16:21.473 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph-authtool: stdout AQD0jOBmHYikLBAAOE3LtlsIF/E0EFQIrr2HLA== 2024-09-10T18:16:22.520 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph-authtool: stdout AQD1jOBm/Z30MxAAURdfXgqm8Bvr5tuKuRnB5A== 2024-09-10T18:16:23.592 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph-authtool: stdout AQD2jOBmmh07NBAAr+VG9WQKWtbLqgijPlotBw== 2024-09-10T18:16:23.593 INFO:teuthology.orchestra.run.smithi115.stdout:Creating initial monmap... 2024-09-10T18:16:24.606 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T18:16:24.607 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-09-10T18:16:24.607 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to a600e180-6fa0-11ef-bcea-c7b262605968 2024-09-10T18:16:24.607 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T18:16:24.607 INFO:teuthology.orchestra.run.smithi115.stdout:monmaptool for a [v2:172.21.15.115:3300,v1:172.21.15.115:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T18:16:24.607 INFO:teuthology.orchestra.run.smithi115.stdout:setting min_mon_release = octopus 2024-09-10T18:16:24.607 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/monmaptool: set fsid to a600e180-6fa0-11ef-bcea-c7b262605968 2024-09-10T18:16:24.607 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T18:16:24.607 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:16:24.607 INFO:teuthology.orchestra.run.smithi115.stdout:Creating mon... 2024-09-10T18:16:25.696 INFO:teuthology.orchestra.run.smithi115.stdout:create mon.a on 2024-09-10T18:16:25.888 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-10T18:16:26.030 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-10T18:16:26.179 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-a600e180-6fa0-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-a600e180-6fa0-11ef-bcea-c7b262605968.target. 2024-09-10T18:16:26.180 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-a600e180-6fa0-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-a600e180-6fa0-11ef-bcea-c7b262605968.target. 2024-09-10T18:16:26.343 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a600e180-6fa0-11ef-bcea-c7b262605968@mon.a 2024-09-10T18:16:26.343 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a600e180-6fa0-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-a600e180-6fa0-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-10T18:16:26.500 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a600e180-6fa0-11ef-bcea-c7b262605968.target.wants/ceph-a600e180-6fa0-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-a600e180-6fa0-11ef-bcea-c7b262605968@.service. 2024-09-10T18:16:27.012 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:26 smithi115 podman[29095]: 2024-09-10 18:16:26.975165072 +0000 UTC m=+0.328601900 container start 9962bf30e8c6de1c9d4c7cb88bd63dec78b5216b29bc70be467811b6a16a4a8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-a, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-10T18:16:27.012 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:27 smithi115 ceph-mon[29132]: mkfs a600e180-6fa0-11ef-bcea-c7b262605968 2024-09-10T18:16:27.085 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T18:16:27.085 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout disabled 2024-09-10T18:16:27.093 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T18:16:27.093 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout inactive 2024-09-10T18:16:27.093 INFO:teuthology.orchestra.run.smithi115.stdout:firewalld.service is not enabled 2024-09-10T18:16:27.093 INFO:teuthology.orchestra.run.smithi115.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T18:16:27.093 INFO:teuthology.orchestra.run.smithi115.stdout:Waiting for mon to start... 2024-09-10T18:16:27.093 INFO:teuthology.orchestra.run.smithi115.stdout:Waiting for mon... 2024-09-10T18:16:27.308 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:27 smithi115 ceph-mon[29132]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T18:16:27.308 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:27 smithi115 bash[29095]: 9962bf30e8c6de1c9d4c7cb88bd63dec78b5216b29bc70be467811b6a16a4a8c 2024-09-10T18:16:27.308 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:27 smithi115 systemd[1]: Started Ceph mon.a for a600e180-6fa0-11ef-bcea-c7b262605968. 2024-09-10T18:16:28.162 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:28 smithi115 ceph-mon[29132]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T18:16:28.162 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:28 smithi115 ceph-mon[29132]: monmap e1: 1 mons at {a=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:16:28.162 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:28 smithi115 ceph-mon[29132]: fsmap 2024-09-10T18:16:28.162 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:28 smithi115 ceph-mon[29132]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T18:16:28.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:28 smithi115 ceph-mon[29132]: mgrmap e1: no daemons active 2024-09-10T18:16:28.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:28 smithi115 ceph-mon[29132]: from='client.? 172.21.15.115:0/471853311' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-10T18:16:28.167 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout cluster: 2024-09-10T18:16:28.167 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout id: a600e180-6fa0-11ef-bcea-c7b262605968 2024-09-10T18:16:28.167 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-10T18:16:28.168 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-09-10T18:16:28.168 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout services: 2024-09-10T18:16:28.168 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.566045s) 2024-09-10T18:16:28.168 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-10T18:16:28.168 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-10T18:16:28.168 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-09-10T18:16:28.168 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout data: 2024-09-10T18:16:28.169 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-10T18:16:28.169 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-10T18:16:28.169 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-10T18:16:28.169 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout pgs: 2024-09-10T18:16:28.169 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-09-10T18:16:28.169 INFO:teuthology.orchestra.run.smithi115.stdout:mon is available 2024-09-10T18:16:28.169 INFO:teuthology.orchestra.run.smithi115.stdout:Assimilating anything we can from ceph.conf... 2024-09-10T18:16:29.204 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-09-10T18:16:29.204 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T18:16:29.205 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout fsid = a600e180-6fa0-11ef-bcea-c7b262605968 2024-09-10T18:16:29.205 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-10T18:16:29.205 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.115:3300,v1:172.21.15.115:6789] 2024-09-10T18:16:29.205 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T18:16:29.205 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T18:16:29.205 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T18:16:29.205 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T18:16:29.205 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-09-10T18:16:29.205 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T18:16:29.205 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T18:16:29.205 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-09-10T18:16:29.205 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T18:16:29.205 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T18:16:29.206 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T18:16:29.206 INFO:teuthology.orchestra.run.smithi115.stdout:Generating new minimal ceph.conf... 2024-09-10T18:16:29.234 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:29 smithi115 ceph-mon[29132]: from='client.? 172.21.15.115:0/3528414525' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T18:16:29.234 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:29 smithi115 ceph-mon[29132]: from='client.? 172.21.15.115:0/3528414525' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-10T18:16:30.289 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:30 smithi115 ceph-mon[29132]: from='client.? 172.21.15.115:0/1448238991' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:16:30.292 INFO:teuthology.orchestra.run.smithi115.stdout:Restarting the monitor... 2024-09-10T18:16:30.570 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:30 smithi115 systemd[1]: Stopping Ceph mon.a for a600e180-6fa0-11ef-bcea-c7b262605968... 2024-09-10T18:16:30.976 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:30 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-a[29109]: 2024-09-10T18:16:30.566+0000 7feef59e0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:16:30.976 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:30 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-a[29109]: 2024-09-10T18:16:30.566+0000 7feef59e0640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-10T18:16:30.976 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:30 smithi115 podman[29448]: 2024-09-10 18:16:30.701051976 +0000 UTC m=+0.235234939 container died 9962bf30e8c6de1c9d4c7cb88bd63dec78b5216b29bc70be467811b6a16a4a8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-10T18:16:31.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 podman[29448]: 2024-09-10 18:16:31.074449342 +0000 UTC m=+0.608632346 container cleanup 9962bf30e8c6de1c9d4c7cb88bd63dec78b5216b29bc70be467811b6a16a4a8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-10T18:16:31.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 bash[29448]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-a 2024-09-10T18:16:31.694 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 podman[29465]: 2024-09-10 18:16:31.333470674 +0000 UTC m=+0.631441130 container remove 9962bf30e8c6de1c9d4c7cb88bd63dec78b5216b29bc70be467811b6a16a4a8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-10T18:16:31.694 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 systemd[1]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-10T18:16:31.694 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 systemd[1]: Stopped Ceph mon.a for a600e180-6fa0-11ef-bcea-c7b262605968. 2024-09-10T18:16:31.694 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 systemd[1]: Starting Ceph mon.a for a600e180-6fa0-11ef-bcea-c7b262605968... 2024-09-10T18:16:31.944 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 podman[29566]: 2024-09-10 18:16:31.69277223 +0000 UTC m=+0.112206211 container create b87dedb747644098ed246821a51bc3b45835b447bd981951c343671da11a7e3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-10T18:16:31.945 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 podman[29566]: 2024-09-10 18:16:31.595926393 +0000 UTC m=+0.015360390 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:16:31.945 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 podman[29566]: 2024-09-10 18:16:31.920307098 +0000 UTC m=+0.339741079 container init b87dedb747644098ed246821a51bc3b45835b447bd981951c343671da11a7e3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-10T18:16:31.945 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 podman[29566]: 2024-09-10 18:16:31.92530192 +0000 UTC m=+0.344735901 container start b87dedb747644098ed246821a51bc3b45835b447bd981951c343671da11a7e3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-a, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-10T18:16:32.042 INFO:teuthology.orchestra.run.smithi115.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-10T18:16:32.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T18:16:32.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable), process ceph-mon, pid 7 2024-09-10T18:16:32.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: pidfile_write: ignore empty --pid-file 2024-09-10T18:16:32.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: load: jerasure load: lrc 2024-09-10T18:16:32.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: RocksDB version: 6.15.5 2024-09-10T18:16:32.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-10T18:16:32.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Compile date Sep 9 2024 2024-09-10T18:16:32.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: DB SUMMARY 2024-09-10T18:16:32.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: DB Session ID: 8Q0B5ANGJJ4V9VEKSB83 2024-09-10T18:16:32.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: CURRENT file: CURRENT 2024-09-10T18:16:32.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T18:16:32.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-09-10T18:16:32.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-10T18:16:32.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000010.log size: 84166 ; 2024-09-10T18:16:32.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.error_if_exists: 0 2024-09-10T18:16:32.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.create_if_missing: 0 2024-09-10T18:16:32.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.paranoid_checks: 1 2024-09-10T18:16:32.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T18:16:32.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.env: 0x55af5c2a9440 2024-09-10T18:16:32.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.fs: Posix File System 2024-09-10T18:16:32.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.info_log: 0x55af5e7baec0 2024-09-10T18:16:32.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T18:16:32.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.statistics: (nil) 2024-09-10T18:16:32.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.use_fsync: 0 2024-09-10T18:16:32.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.max_log_file_size: 0 2024-09-10T18:16:32.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T18:16:32.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T18:16:32.238 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T18:16:32.238 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T18:16:32.238 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.allow_fallocate: 1 2024-09-10T18:16:32.238 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T18:16:32.238 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T18:16:32.238 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.use_direct_reads: 0 2024-09-10T18:16:32.238 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T18:16:32.238 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T18:16:32.238 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.db_log_dir: 2024-09-10T18:16:32.238 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-09-10T18:16:32.238 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T18:16:32.238 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T18:16:32.238 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T18:16:32.239 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T18:16:32.239 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T18:16:32.239 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T18:16:32.239 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T18:16:32.239 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T18:16:32.239 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.write_buffer_manager: 0x55af5e878780 2024-09-10T18:16:32.239 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T18:16:32.239 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-10T18:16:32.239 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T18:16:32.239 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T18:16:32.239 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T18:16:32.239 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T18:16:32.239 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T18:16:32.240 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T18:16:32.240 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T18:16:32.240 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.unordered_write: 0 2024-09-10T18:16:32.240 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T18:16:32.240 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T18:16:32.240 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T18:16:32.240 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T18:16:32.240 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.row_cache: None 2024-09-10T18:16:32.240 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.wal_filter: None 2024-09-10T18:16:32.240 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T18:16:32.240 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T18:16:32.240 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.preserve_deletes: 0 2024-09-10T18:16:32.241 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.two_write_queues: 0 2024-09-10T18:16:32.241 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T18:16:32.241 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.atomic_flush: 0 2024-09-10T18:16:32.241 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T18:16:32.241 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T18:16:32.241 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T18:16:32.241 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.log_readahead_size: 0 2024-09-10T18:16:32.241 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T18:16:32.241 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T18:16:32.241 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T18:16:32.241 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T18:16:32.241 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T18:16:32.241 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T18:16:32.242 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.max_background_jobs: 2 2024-09-10T18:16:32.243 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.max_background_compactions: -1 2024-09-10T18:16:32.243 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.max_subcompactions: 1 2024-09-10T18:16:32.243 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T18:16:32.243 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T18:16:32.243 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T18:16:32.243 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T18:16:32.243 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T18:16:32.243 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T18:16:32.243 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T18:16:32.243 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T18:16:32.244 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.max_open_files: -1 2024-09-10T18:16:32.244 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T18:16:32.244 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T18:16:32.244 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T18:16:32.244 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T18:16:32.244 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.max_background_flushes: -1 2024-09-10T18:16:32.244 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Compression algorithms supported: 2024-09-10T18:16:32.244 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: kZSTD supported: 0 2024-09-10T18:16:32.244 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: kXpressCompression supported: 0 2024-09-10T18:16:32.244 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: kBZip2Compression supported: 0 2024-09-10T18:16:32.244 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T18:16:32.244 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: kLZ4Compression supported: 1 2024-09-10T18:16:32.245 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: kZlibCompression supported: 1 2024-09-10T18:16:32.245 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T18:16:32.245 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: kSnappyCompression supported: 1 2024-09-10T18:16:32.245 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T18:16:32.245 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 2024-09-10T18:16:32.245 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-10T18:16:32.245 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T18:16:32.245 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.merge_operator: 2024-09-10T18:16:32.245 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.compaction_filter: None 2024-09-10T18:16:32.245 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T18:16:32.245 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T18:16:32.245 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T18:16:32.245 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T18:16:32.246 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55af5e761a38) 2024-09-10T18:16:32.246 INFO:journalctl@ceph.mon.a.smithi115.stdout: cache_index_and_filter_blocks: 1 2024-09-10T18:16:32.246 INFO:journalctl@ceph.mon.a.smithi115.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T18:16:32.246 INFO:journalctl@ceph.mon.a.smithi115.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T18:16:32.246 INFO:journalctl@ceph.mon.a.smithi115.stdout: pin_top_level_index_and_filter: 1 2024-09-10T18:16:32.246 INFO:journalctl@ceph.mon.a.smithi115.stdout: index_type: 0 2024-09-10T18:16:32.246 INFO:journalctl@ceph.mon.a.smithi115.stdout: data_block_index_type: 0 2024-09-10T18:16:32.246 INFO:journalctl@ceph.mon.a.smithi115.stdout: index_shortening: 1 2024-09-10T18:16:32.246 INFO:journalctl@ceph.mon.a.smithi115.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T18:16:32.246 INFO:journalctl@ceph.mon.a.smithi115.stdout: hash_index_allow_collision: 1 2024-09-10T18:16:32.246 INFO:journalctl@ceph.mon.a.smithi115.stdout: checksum: 1 2024-09-10T18:16:32.246 INFO:journalctl@ceph.mon.a.smithi115.stdout: no_block_cache: 0 2024-09-10T18:16:32.246 INFO:journalctl@ceph.mon.a.smithi115.stdout: block_cache: 0x55af5e7971f0 2024-09-10T18:16:32.246 INFO:journalctl@ceph.mon.a.smithi115.stdout: block_cache_name: BinnedLRUCache 2024-09-10T18:16:32.247 INFO:journalctl@ceph.mon.a.smithi115.stdout: block_cache_options: 2024-09-10T18:16:32.247 INFO:journalctl@ceph.mon.a.smithi115.stdout: capacity : 536870912 2024-09-10T18:16:32.247 INFO:journalctl@ceph.mon.a.smithi115.stdout: num_shard_bits : 4 2024-09-10T18:16:32.247 INFO:journalctl@ceph.mon.a.smithi115.stdout: strict_capacity_limit : 0 2024-09-10T18:16:32.247 INFO:journalctl@ceph.mon.a.smithi115.stdout: high_pri_pool_ratio: 0.000 2024-09-10T18:16:32.247 INFO:journalctl@ceph.mon.a.smithi115.stdout: block_cache_compressed: (nil) 2024-09-10T18:16:32.247 INFO:journalctl@ceph.mon.a.smithi115.stdout: persistent_cache: (nil) 2024-09-10T18:16:32.247 INFO:journalctl@ceph.mon.a.smithi115.stdout: block_size: 4096 2024-09-10T18:16:32.247 INFO:journalctl@ceph.mon.a.smithi115.stdout: block_size_deviation: 10 2024-09-10T18:16:32.247 INFO:journalctl@ceph.mon.a.smithi115.stdout: block_restart_interval: 16 2024-09-10T18:16:32.247 INFO:journalctl@ceph.mon.a.smithi115.stdout: index_block_restart_interval: 1 2024-09-10T18:16:32.247 INFO:journalctl@ceph.mon.a.smithi115.stdout: metadata_block_size: 4096 2024-09-10T18:16:32.247 INFO:journalctl@ceph.mon.a.smithi115.stdout: partition_filters: 0 2024-09-10T18:16:32.247 INFO:journalctl@ceph.mon.a.smithi115.stdout: use_delta_encoding: 1 2024-09-10T18:16:32.248 INFO:journalctl@ceph.mon.a.smithi115.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-10T18:16:32.248 INFO:journalctl@ceph.mon.a.smithi115.stdout: whole_key_filtering: 1 2024-09-10T18:16:32.248 INFO:journalctl@ceph.mon.a.smithi115.stdout: verify_compression: 0 2024-09-10T18:16:32.248 INFO:journalctl@ceph.mon.a.smithi115.stdout: read_amp_bytes_per_bit: 0 2024-09-10T18:16:32.248 INFO:journalctl@ceph.mon.a.smithi115.stdout: format_version: 4 2024-09-10T18:16:32.248 INFO:journalctl@ceph.mon.a.smithi115.stdout: enable_index_compression: 1 2024-09-10T18:16:32.248 INFO:journalctl@ceph.mon.a.smithi115.stdout: block_align: 0 2024-09-10T18:16:32.248 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T18:16:32.248 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T18:16:32.248 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.compression: NoCompression 2024-09-10T18:16:32.248 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T18:16:32.248 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T18:16:32.248 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T18:16:32.249 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.num_levels: 7 2024-09-10T18:16:32.249 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T18:16:32.249 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T18:16:32.249 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T18:16:32.249 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T18:16:32.249 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T18:16:32.249 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T18:16:32.249 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T18:16:32.249 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:16:32.249 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T18:16:32.249 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T18:16:32.249 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T18:16:32.249 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T18:16:32.249 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T18:16:32.250 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T18:16:32.250 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:16:32.250 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T18:16:32.250 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T18:16:32.250 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T18:16:32.250 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T18:16:32.250 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T18:16:32.250 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T18:16:32.250 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T18:16:32.250 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T18:16:32.250 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T18:16:32.250 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T18:16:32.250 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T18:16:32.250 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T18:16:32.251 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T18:16:32.251 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T18:16:32.251 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T18:16:32.251 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T18:16:32.251 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T18:16:32.251 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T18:16:32.251 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T18:16:32.251 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.arena_block_size: 4194304 2024-09-10T18:16:32.251 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T18:16:32.251 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T18:16:32.251 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-10T18:16:32.251 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T18:16:32.251 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T18:16:32.252 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T18:16:32.252 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T18:16:32.252 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T18:16:32.252 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T18:16:32.252 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T18:16:32.252 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T18:16:32.252 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T18:16:32.252 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T18:16:32.252 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T18:16:32.252 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T18:16:32.252 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.inplace_update_support: 0 2024-09-10T18:16:32.252 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T18:16:32.252 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T18:16:32.252 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T18:16:32.253 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T18:16:32.253 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.bloom_locality: 0 2024-09-10T18:16:32.253 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.max_successive_merges: 0 2024-09-10T18:16:32.253 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T18:16:32.253 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T18:16:32.253 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T18:16:32.253 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T18:16:32.253 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.ttl: 2592000 2024-09-10T18:16:32.253 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T18:16:32.253 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.enable_blob_files: false 2024-09-10T18:16:32.253 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.min_blob_size: 0 2024-09-10T18:16:32.253 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T18:16:32.254 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T18:16:32.254 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T18:16:32.254 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T18:16:32.254 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T18:16:32.254 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-09-10T18:16:32.254 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-09-10T18:16:32.254 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725992191948587, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-09-10T18:16:32.254 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-09-10T18:16:32.254 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-10T18:16:32.254 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725992191949295, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 80753, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 78992, "index_size": 223, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10080, "raw_average_key_size": 47, "raw_value_size": 73244, "raw_average_value_size": 343, "num_data_blocks": 10, "num_entries": 213, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1725992191, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "469e2459-1626-4b84-ad51-1d550e9d4a61", "db_session_id": "8Q0B5ANGJJ4V9VEKSB83"}} 2024-09-10T18:16:32.254 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-09-10T18:16:32.254 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725992191949666, "job": 1, "event": "recovery_finished"} 2024-09-10T18:16:32.254 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T18:16:32.255 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55af5e7ca700 2024-09-10T18:16:32.255 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: DB pointer 0x55af5e79a000 2024-09-10T18:16:32.255 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-10T18:16:32.255 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-10T18:16:32.255 INFO:journalctl@ceph.mon.a.smithi115.stdout: ** DB Stats ** 2024-09-10T18:16:32.255 INFO:journalctl@ceph.mon.a.smithi115.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:16:32.255 INFO:journalctl@ceph.mon.a.smithi115.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T18:16:32.255 INFO:journalctl@ceph.mon.a.smithi115.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T18:16:32.255 INFO:journalctl@ceph.mon.a.smithi115.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:16:32.255 INFO:journalctl@ceph.mon.a.smithi115.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T18:16:32.255 INFO:journalctl@ceph.mon.a.smithi115.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-10T18:16:32.255 INFO:journalctl@ceph.mon.a.smithi115.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:16:32.255 INFO:journalctl@ceph.mon.a.smithi115.stdout: 2024-09-10T18:16:32.255 INFO:journalctl@ceph.mon.a.smithi115.stdout: ** Compaction Stats [default] ** 2024-09-10T18:16:32.256 INFO:journalctl@ceph.mon.a.smithi115.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:16:32.256 INFO:journalctl@ceph.mon.a.smithi115.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:16:32.256 INFO:journalctl@ceph.mon.a.smithi115.stdout: L0 2/0 80.65 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 163.2 0.00 0.00 1 0.000 0 0 2024-09-10T18:16:32.256 INFO:journalctl@ceph.mon.a.smithi115.stdout: Sum 2/0 80.65 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 163.2 0.00 0.00 1 0.000 0 0 2024-09-10T18:16:32.256 INFO:journalctl@ceph.mon.a.smithi115.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 163.2 0.00 0.00 1 0.000 0 0 2024-09-10T18:16:32.256 INFO:journalctl@ceph.mon.a.smithi115.stdout: 2024-09-10T18:16:32.256 INFO:journalctl@ceph.mon.a.smithi115.stdout: ** Compaction Stats [default] ** 2024-09-10T18:16:32.256 INFO:journalctl@ceph.mon.a.smithi115.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:16:32.256 INFO:journalctl@ceph.mon.a.smithi115.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:16:32.256 INFO:journalctl@ceph.mon.a.smithi115.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 163.2 0.00 0.00 1 0.000 0 0 2024-09-10T18:16:32.256 INFO:journalctl@ceph.mon.a.smithi115.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:16:32.256 INFO:journalctl@ceph.mon.a.smithi115.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:16:32.257 INFO:journalctl@ceph.mon.a.smithi115.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:16:32.257 INFO:journalctl@ceph.mon.a.smithi115.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:16:32.257 INFO:journalctl@ceph.mon.a.smithi115.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:16:32.257 INFO:journalctl@ceph.mon.a.smithi115.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:16:32.257 INFO:journalctl@ceph.mon.a.smithi115.stdout: Cumulative compaction: 0.00 GB write, 25.23 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:16:32.257 INFO:journalctl@ceph.mon.a.smithi115.stdout: Interval compaction: 0.00 GB write, 25.23 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:16:32.257 INFO:journalctl@ceph.mon.a.smithi115.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:16:32.257 INFO:journalctl@ceph.mon.a.smithi115.stdout: 2024-09-10T18:16:32.257 INFO:journalctl@ceph.mon.a.smithi115.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:16:32.257 INFO:journalctl@ceph.mon.a.smithi115.stdout: 2024-09-10T18:16:32.257 INFO:journalctl@ceph.mon.a.smithi115.stdout: ** Compaction Stats [default] ** 2024-09-10T18:16:32.257 INFO:journalctl@ceph.mon.a.smithi115.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:16:32.257 INFO:journalctl@ceph.mon.a.smithi115.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:16:32.258 INFO:journalctl@ceph.mon.a.smithi115.stdout: L0 2/0 80.65 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 163.2 0.00 0.00 1 0.000 0 0 2024-09-10T18:16:32.258 INFO:journalctl@ceph.mon.a.smithi115.stdout: Sum 2/0 80.65 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 163.2 0.00 0.00 1 0.000 0 0 2024-09-10T18:16:32.258 INFO:journalctl@ceph.mon.a.smithi115.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-10T18:16:32.258 INFO:journalctl@ceph.mon.a.smithi115.stdout: 2024-09-10T18:16:32.258 INFO:journalctl@ceph.mon.a.smithi115.stdout: ** Compaction Stats [default] ** 2024-09-10T18:16:32.258 INFO:journalctl@ceph.mon.a.smithi115.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:16:32.258 INFO:journalctl@ceph.mon.a.smithi115.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:16:32.258 INFO:journalctl@ceph.mon.a.smithi115.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 163.2 0.00 0.00 1 0.000 0 0 2024-09-10T18:16:32.258 INFO:journalctl@ceph.mon.a.smithi115.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:16:32.258 INFO:journalctl@ceph.mon.a.smithi115.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:16:32.258 INFO:journalctl@ceph.mon.a.smithi115.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:16:32.258 INFO:journalctl@ceph.mon.a.smithi115.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:16:32.258 INFO:journalctl@ceph.mon.a.smithi115.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:16:32.258 INFO:journalctl@ceph.mon.a.smithi115.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:16:32.259 INFO:journalctl@ceph.mon.a.smithi115.stdout: Cumulative compaction: 0.00 GB write, 24.97 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:16:32.259 INFO:journalctl@ceph.mon.a.smithi115.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:16:32.259 INFO:journalctl@ceph.mon.a.smithi115.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:16:32.259 INFO:journalctl@ceph.mon.a.smithi115.stdout: 2024-09-10T18:16:32.259 INFO:journalctl@ceph.mon.a.smithi115.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:16:32.259 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: starting mon.a rank 0 at public addrs [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] at bind addrs [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid a600e180-6fa0-11ef-bcea-c7b262605968 2024-09-10T18:16:32.259 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: mon.a@-1(???) e1 preinit fsid a600e180-6fa0-11ef-bcea-c7b262605968 2024-09-10T18:16:32.259 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: mon.a@-1(???).mds e1 new map 2024-09-10T18:16:32.259 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: mon.a@-1(???).mds e1 print_map 2024-09-10T18:16:32.259 INFO:journalctl@ceph.mon.a.smithi115.stdout: e1 2024-09-10T18:16:32.259 INFO:journalctl@ceph.mon.a.smithi115.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T18:16:32.259 INFO:journalctl@ceph.mon.a.smithi115.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T18:16:32.259 INFO:journalctl@ceph.mon.a.smithi115.stdout: legacy client fscid: -1 2024-09-10T18:16:32.259 INFO:journalctl@ceph.mon.a.smithi115.stdout: 2024-09-10T18:16:32.260 INFO:journalctl@ceph.mon.a.smithi115.stdout: No filesystems configured 2024-09-10T18:16:32.260 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T18:16:32.260 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:16:32.260 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:16:32.260 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:16:32.260 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:31 smithi115 ceph-mon[29603]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-10T18:16:32.260 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:32 smithi115 ceph-mon[29603]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T18:16:32.260 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:32 smithi115 ceph-mon[29603]: monmap e1: 1 mons at {a=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:16:32.260 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:32 smithi115 ceph-mon[29603]: fsmap 2024-09-10T18:16:32.260 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:32 smithi115 ceph-mon[29603]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T18:16:32.260 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:32 smithi115 ceph-mon[29603]: mgrmap e1: no daemons active 2024-09-10T18:16:32.260 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:32 smithi115 bash[29566]: b87dedb747644098ed246821a51bc3b45835b447bd981951c343671da11a7e3f 2024-09-10T18:16:32.260 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:32 smithi115 systemd[1]: Started Ceph mon.a for a600e180-6fa0-11ef-bcea-c7b262605968. 2024-09-10T18:16:33.114 INFO:teuthology.orchestra.run.smithi115.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-10T18:16:33.114 INFO:teuthology.orchestra.run.smithi115.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:16:33.114 INFO:teuthology.orchestra.run.smithi115.stdout:Creating mgr... 2024-09-10T18:16:33.115 INFO:teuthology.orchestra.run.smithi115.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-10T18:16:33.115 INFO:teuthology.orchestra.run.smithi115.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-10T18:16:33.294 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a600e180-6fa0-11ef-bcea-c7b262605968@mgr.y 2024-09-10T18:16:33.295 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a600e180-6fa0-11ef-bcea-c7b262605968@mgr.y.service: Unit ceph-a600e180-6fa0-11ef-bcea-c7b262605968@mgr.y.service not loaded. 2024-09-10T18:16:33.443 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a600e180-6fa0-11ef-bcea-c7b262605968.target.wants/ceph-a600e180-6fa0-11ef-bcea-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-a600e180-6fa0-11ef-bcea-c7b262605968@.service. 2024-09-10T18:16:33.463 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:33 smithi115 systemd[1]: Starting Ceph mgr.y for a600e180-6fa0-11ef-bcea-c7b262605968... 2024-09-10T18:16:33.734 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:33 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/3167860486' entity='client.admin' 2024-09-10T18:16:33.735 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:33 smithi115 podman[29839]: 2024-09-10 18:16:33.587234752 +0000 UTC m=+0.028586307 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:16:33.735 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:33 smithi115 podman[29839]: 2024-09-10 18:16:33.693982462 +0000 UTC m=+0.135334054 container create bcf84116aaa96ed215eb03dc40d579f6c756560a6abb2f16801f553a469b4598 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:16:34.045 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T18:16:34.045 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout disabled 2024-09-10T18:16:34.052 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T18:16:34.053 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout inactive 2024-09-10T18:16:34.053 INFO:teuthology.orchestra.run.smithi115.stdout:firewalld.service is not enabled 2024-09-10T18:16:34.053 INFO:teuthology.orchestra.run.smithi115.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T18:16:34.061 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T18:16:34.062 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout disabled 2024-09-10T18:16:34.070 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T18:16:34.070 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout inactive 2024-09-10T18:16:34.070 INFO:teuthology.orchestra.run.smithi115.stdout:firewalld.service is not enabled 2024-09-10T18:16:34.070 INFO:teuthology.orchestra.run.smithi115.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-10T18:16:34.070 INFO:teuthology.orchestra.run.smithi115.stdout:Waiting for mgr to start... 2024-09-10T18:16:34.070 INFO:teuthology.orchestra.run.smithi115.stdout:Waiting for mgr... 2024-09-10T18:16:34.195 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:33 smithi115 podman[29839]: 2024-09-10 18:16:33.921289773 +0000 UTC m=+0.362641319 container init bcf84116aaa96ed215eb03dc40d579f6c756560a6abb2f16801f553a469b4598 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-10T18:16:34.195 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:33 smithi115 podman[29839]: 2024-09-10 18:16:33.926372492 +0000 UTC m=+0.367724039 container start bcf84116aaa96ed215eb03dc40d579f6c756560a6abb2f16801f553a469b4598 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:16:34.196 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:34 smithi115 bash[29839]: bcf84116aaa96ed215eb03dc40d579f6c756560a6abb2f16801f553a469b4598 2024-09-10T18:16:34.196 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:34 smithi115 systemd[1]: Started Ceph mgr.y for a600e180-6fa0-11ef-bcea-c7b262605968. 2024-09-10T18:16:34.196 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:34 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:34.038+0000 7f205285fec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:16:34.196 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:34 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:34.097+0000 7f205285fec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:16:34.196 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:34 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:34.162+0000 7f205285fec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:16:34.485 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:34 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:34.288+0000 7f205285fec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:16:34.485 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:34 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:34.430+0000 7f205285fec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:16:34.837 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:34 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1186862172' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T18:16:34.837 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:34 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:34.700+0000 7f205285fec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:16:34.837 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:34 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:34.762+0000 7f205285fec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:16:35.137 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:34 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:34.927+0000 7f205285fec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:16:35.146 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-09-10T18:16:35.146 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:16:35.146 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "fsid": "a600e180-6fa0-11ef-bcea-c7b262605968", 2024-09-10T18:16:35.146 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T18:16:35.146 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T18:16:35.146 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T18:16:35.147 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T18:16:35.147 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:16:35.147 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T18:16:35.147 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T18:16:35.147 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 0 2024-09-10T18:16:35.147 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:16:35.147 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T18:16:35.147 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T18:16:35.147 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:16:35.147 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-10T18:16:35.147 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T18:16:35.147 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:16:35.148 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-10T18:16:35.148 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T18:16:35.148 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:16:35.148 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T18:16:35.148 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:16:35.148 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T18:16:35.148 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T18:16:35.148 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T18:16:35.148 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T18:16:35.148 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T18:16:35.148 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T18:16:35.149 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:16:35.149 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T18:16:35.149 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T18:16:35.149 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T18:16:35.149 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T18:16:35.149 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T18:16:35.149 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T18:16:35.149 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T18:16:35.149 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T18:16:35.149 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T18:16:35.149 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:16:35.149 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T18:16:35.150 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:16:35.150 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T18:16:35.150 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T18:16:35.150 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:16:35.150 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T18:16:35.150 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T18:16:35.150 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T18:16:35.150 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T18:16:35.150 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T18:16:35.150 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T18:16:35.150 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T18:16:35.150 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:16:35.151 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:16:35.151 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:16:35.151 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T18:16:35.151 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:16:35.151 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T18:16:27.008838+0000", 2024-09-10T18:16:35.151 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:16:35.151 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:16:35.151 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T18:16:35.151 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:16:35.151 INFO:teuthology.orchestra.run.smithi115.stdout:mgr not available, waiting (1/15)... 2024-09-10T18:16:36.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:35 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:35.769+0000 7f205285fec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:16:36.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:35 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:35.833+0000 7f205285fec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:16:36.827 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:36 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:36.537+0000 7f205285fec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:16:36.828 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:36 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:36.646+0000 7f205285fec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:16:36.828 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:36 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:36.766+0000 7f205285fec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:16:37.215 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:36 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:36.824+0000 7f205285fec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:16:37.215 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:36 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:36.948+0000 7f205285fec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:16:37.215 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:37 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:37.014+0000 7f205285fec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:16:37.215 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:37 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:37.071+0000 7f205285fec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:16:37.216 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:37 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:37.212+0000 7f205285fec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:16:37.485 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:37 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:37.275+0000 7f205285fec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:16:37.485 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:37 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:37.341+0000 7f205285fec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:16:37.485 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:37 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:37.425+0000 7f205285fec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:16:37.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:37 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/3986353831' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T18:16:37.986 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:37 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:37.484+0000 7f205285fec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:16:37.986 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:37 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:37.540+0000 7f205285fec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:16:38.290 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-09-10T18:16:38.290 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:16:38.290 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "fsid": "a600e180-6fa0-11ef-bcea-c7b262605968", 2024-09-10T18:16:38.290 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T18:16:38.290 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T18:16:38.291 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T18:16:38.291 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T18:16:38.291 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:16:38.291 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T18:16:38.291 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T18:16:38.291 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 0 2024-09-10T18:16:38.291 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:16:38.291 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T18:16:38.291 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T18:16:38.291 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:16:38.291 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-10T18:16:38.291 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T18:16:38.291 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:16:38.292 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-10T18:16:38.292 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T18:16:38.292 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:16:38.292 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T18:16:38.292 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:16:38.292 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T18:16:38.292 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T18:16:38.292 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T18:16:38.292 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T18:16:38.292 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T18:16:38.292 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T18:16:38.295 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:16:38.296 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T18:16:38.296 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T18:16:38.296 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T18:16:38.296 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T18:16:38.296 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T18:16:38.296 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T18:16:38.296 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T18:16:38.296 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T18:16:38.296 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T18:16:38.296 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:16:38.296 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T18:16:38.296 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:16:38.296 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T18:16:38.297 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T18:16:38.297 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:16:38.297 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T18:16:38.297 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T18:16:38.297 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T18:16:38.297 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T18:16:38.297 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T18:16:38.297 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T18:16:38.297 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T18:16:38.297 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:16:38.297 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:16:38.297 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:16:38.297 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T18:16:38.298 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:16:38.298 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T18:16:27.008838+0000", 2024-09-10T18:16:38.298 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:16:38.298 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:16:38.298 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T18:16:38.298 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:16:38.298 INFO:teuthology.orchestra.run.smithi115.stdout:mgr not available, waiting (2/15)... 2024-09-10T18:16:38.734 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:38 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:38.297+0000 7f205285fec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:16:38.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:38 smithi115 ceph-mon[29603]: Activating manager daemon y 2024-09-10T18:16:38.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:38 smithi115 ceph-mon[29603]: mgrmap e2: y(active, starting, since 0.00272095s) 2024-09-10T18:16:38.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:38 smithi115 ceph-mon[29603]: from='mgr.14100 172.21.15.115:0/2749774997' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:16:38.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:38 smithi115 ceph-mon[29603]: from='mgr.14100 172.21.15.115:0/2749774997' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:16:38.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:38 smithi115 ceph-mon[29603]: from='mgr.14100 172.21.15.115:0/2749774997' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:16:38.986 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:38 smithi115 ceph-mon[29603]: from='mgr.14100 172.21.15.115:0/2749774997' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:16:38.986 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:38 smithi115 ceph-mon[29603]: from='mgr.14100 172.21.15.115:0/2749774997' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:16:38.986 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:38 smithi115 ceph-mon[29603]: Manager daemon y is now available 2024-09-10T18:16:38.986 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:38 smithi115 ceph-mon[29603]: from='mgr.14100 172.21.15.115:0/2749774997' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:16:38.986 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:38 smithi115 ceph-mon[29603]: from='mgr.14100 172.21.15.115:0/2749774997' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:16:38.986 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:38 smithi115 ceph-mon[29603]: from='mgr.14100 172.21.15.115:0/2749774997' entity='mgr.y' 2024-09-10T18:16:38.986 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:38 smithi115 ceph-mon[29603]: from='mgr.14100 172.21.15.115:0/2749774997' entity='mgr.y' 2024-09-10T18:16:38.986 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:38 smithi115 ceph-mon[29603]: from='mgr.14100 172.21.15.115:0/2749774997' entity='mgr.y' 2024-09-10T18:16:40.635 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:40 smithi115 ceph-mon[29603]: mgrmap e3: y(active, since 1.00662s) 2024-09-10T18:16:41.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:41 smithi115 ceph-mon[29603]: mgrmap e4: y(active, since 2s) 2024-09-10T18:16:41.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:41 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/942350915' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T18:16:44.280 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-09-10T18:16:44.280 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:16:44.280 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "fsid": "a600e180-6fa0-11ef-bcea-c7b262605968", 2024-09-10T18:16:44.280 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T18:16:44.280 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T18:16:44.280 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T18:16:44.281 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T18:16:44.281 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:16:44.281 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T18:16:44.281 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T18:16:44.281 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 0 2024-09-10T18:16:44.281 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:16:44.281 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T18:16:44.281 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T18:16:44.281 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:16:44.281 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-10T18:16:44.281 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T18:16:44.281 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:16:44.281 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-10T18:16:44.281 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T18:16:44.282 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:16:44.282 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T18:16:44.282 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:16:44.282 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T18:16:44.282 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T18:16:44.282 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T18:16:44.282 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T18:16:44.282 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T18:16:44.283 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T18:16:44.283 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:16:44.283 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T18:16:44.283 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T18:16:44.283 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T18:16:44.283 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T18:16:44.283 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T18:16:44.283 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T18:16:44.283 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T18:16:44.283 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T18:16:44.283 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T18:16:44.283 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:16:44.283 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T18:16:44.283 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:16:44.284 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T18:16:44.284 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T18:16:44.284 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:16:44.284 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T18:16:44.284 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T18:16:44.284 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T18:16:44.284 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T18:16:44.284 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T18:16:44.284 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T18:16:44.284 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T18:16:44.284 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:16:44.284 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:16:44.285 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:16:44.285 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T18:16:44.285 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:16:44.285 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T18:16:27.008838+0000", 2024-09-10T18:16:44.285 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:16:44.285 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:16:44.285 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T18:16:44.285 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:16:44.285 INFO:teuthology.orchestra.run.smithi115.stdout:mgr is available 2024-09-10T18:16:47.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:47 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1305417145' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T18:16:54.027 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-09-10T18:16:54.027 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T18:16:54.027 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout fsid = a600e180-6fa0-11ef-bcea-c7b262605968 2024-09-10T18:16:54.027 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-10T18:16:54.027 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T18:16:54.027 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T18:16:54.027 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T18:16:54.027 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T18:16:54.027 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-09-10T18:16:54.028 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T18:16:54.028 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T18:16:54.028 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-09-10T18:16:54.028 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T18:16:54.028 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T18:16:54.028 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T18:16:54.028 INFO:teuthology.orchestra.run.smithi115.stdout:Enabling cephadm module... 2024-09-10T18:16:55.024 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:54 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/834902381' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-10T18:16:55.025 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:54 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ignoring --setuser ceph since I am not root 2024-09-10T18:16:55.025 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:54 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ignoring --setgroup ceph since I am not root 2024-09-10T18:16:55.025 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:54 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:54.903+0000 7f4ee6d98ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:16:55.025 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:54 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:54.957+0000 7f4ee6d98ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:16:55.025 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:55 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:55.022+0000 7f4ee6d98ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:16:55.284 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:55 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:55.146+0000 7f4ee6d98ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:16:55.535 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:55 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:55.282+0000 7f4ee6d98ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:16:55.958 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:55 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/834902381' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-10T18:16:55.958 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:55 smithi115 ceph-mon[29603]: mgrmap e5: y(active, since 16s) 2024-09-10T18:16:55.958 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:55 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:55.546+0000 7f4ee6d98ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:16:55.958 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:55 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:55.608+0000 7f4ee6d98ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:16:55.959 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:55 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:55.774+0000 7f4ee6d98ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:16:56.516 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:16:56.517 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-10T18:16:56.517 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T18:16:56.517 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-10T18:16:56.517 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T18:16:56.517 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:16:56.517 INFO:teuthology.orchestra.run.smithi115.stdout:Waiting for the mgr to restart... 2024-09-10T18:16:56.517 INFO:teuthology.orchestra.run.smithi115.stdout:Waiting for mgr epoch 5... 2024-09-10T18:16:56.883 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:56 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/3367705146' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T18:16:56.883 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:56 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:56.630+0000 7f4ee6d98ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:16:56.883 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:56 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:56.695+0000 7f4ee6d98ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:16:57.686 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:57 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:57.399+0000 7f4ee6d98ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:16:57.686 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:57 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:57.507+0000 7f4ee6d98ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:16:57.686 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:57 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:57.626+0000 7f4ee6d98ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:16:57.985 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:57 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:57.684+0000 7f4ee6d98ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:16:57.985 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:57 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:57.806+0000 7f4ee6d98ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:16:57.985 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:57 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:57.871+0000 7f4ee6d98ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:16:57.985 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:57 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:57.928+0000 7f4ee6d98ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:16:58.320 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:58 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:58.059+0000 7f4ee6d98ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:16:58.321 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:58 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:58.121+0000 7f4ee6d98ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:16:58.321 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:58 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:58.184+0000 7f4ee6d98ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:16:58.321 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:58 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:58.262+0000 7f4ee6d98ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:16:58.321 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:58 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:58.318+0000 7f4ee6d98ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:16:58.735 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:58 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:58.375+0000 7f4ee6d98ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:16:59.485 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:16:59 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:16:59.105+0000 7f4ee6d98ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:16:59.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:59 smithi115 ceph-mon[29603]: Active manager daemon y restarted 2024-09-10T18:16:59.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:59 smithi115 ceph-mon[29603]: Activating manager daemon y 2024-09-10T18:16:59.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:59 smithi115 ceph-mon[29603]: osdmap e2: 0 total, 0 up, 0 in 2024-09-10T18:16:59.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:59 smithi115 ceph-mon[29603]: mgrmap e6: y(active, starting, since 0.00309249s) 2024-09-10T18:16:59.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:59 smithi115 ceph-mon[29603]: from='mgr.14118 172.21.15.115:0/680636684' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:16:59.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:59 smithi115 ceph-mon[29603]: from='mgr.14118 172.21.15.115:0/680636684' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:16:59.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:59 smithi115 ceph-mon[29603]: from='mgr.14118 172.21.15.115:0/680636684' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:16:59.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:59 smithi115 ceph-mon[29603]: from='mgr.14118 172.21.15.115:0/680636684' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:16:59.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:59 smithi115 ceph-mon[29603]: from='mgr.14118 172.21.15.115:0/680636684' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:16:59.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:59 smithi115 ceph-mon[29603]: Manager daemon y is now available 2024-09-10T18:16:59.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:59 smithi115 ceph-mon[29603]: from='mgr.14118 172.21.15.115:0/680636684' entity='mgr.y' 2024-09-10T18:16:59.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:59 smithi115 ceph-mon[29603]: from='mgr.14118 172.21.15.115:0/680636684' entity='mgr.y' 2024-09-10T18:16:59.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:59 smithi115 ceph-mon[29603]: from='mgr.14118 172.21.15.115:0/680636684' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:16:59.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:59 smithi115 ceph-mon[29603]: from='mgr.14118 172.21.15.115:0/680636684' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:16:59.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:59 smithi115 ceph-mon[29603]: from='mgr.14118 172.21.15.115:0/680636684' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:16:59.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:16:59 smithi115 ceph-mon[29603]: from='mgr.14118 172.21.15.115:0/680636684' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:17:00.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:00 smithi115 ceph-mon[29603]: Found migration_current of "None". Setting to last migration. 2024-09-10T18:17:00.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:00 smithi115 ceph-mon[29603]: mgrmap e7: y(active, since 1.00504s) 2024-09-10T18:17:00.828 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:17:00.557+0000 7f4e99186640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:18:17:00] ENGINE Error in HTTPServer.serve 2024-09-10T18:17:00.828 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: Traceback (most recent call last): 2024-09-10T18:17:00.828 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:17:00.828 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: self._connections.run(self.expiration_interval) 2024-09-10T18:17:00.828 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:17:00.828 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: self._run(expiration_interval) 2024-09-10T18:17:00.829 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:17:00.829 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:17:00.829 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:17:00.829 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:17:00.829 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:17:00.829 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: s = self.context.wrap_socket( 2024-09-10T18:17:00.829 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:17:00.829 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: return self.sslsocket_class._create( 2024-09-10T18:17:00.829 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:17:00.829 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: self.do_handshake() 2024-09-10T18:17:00.829 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:17:00.829 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: self._sslobj.do_handshake() 2024-09-10T18:17:00.829 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:17:00.829 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:17:00.830 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:17:00.830 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-10T18:17:00.830 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T18:17:00.831 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:17:00.831 INFO:teuthology.orchestra.run.smithi115.stdout:mgr epoch 5 is available 2024-09-10T18:17:00.831 INFO:teuthology.orchestra.run.smithi115.stdout:Setting orchestrator backend to cephadm... 2024-09-10T18:17:01.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:01 smithi115 ceph-mon[29603]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T18:17:01.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:01 smithi115 ceph-mon[29603]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T18:17:01.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:01 smithi115 ceph-mon[29603]: [10/Sep/2024:18:17:00] ENGINE Bus STARTING 2024-09-10T18:17:01.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:01 smithi115 ceph-mon[29603]: [10/Sep/2024:18:17:00] ENGINE Serving on https://172.21.15.115:7150 2024-09-10T18:17:01.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:01 smithi115 ceph-mon[29603]: [10/Sep/2024:18:17:00] ENGINE Bus STARTED 2024-09-10T18:17:01.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:01 smithi115 ceph-mon[29603]: [10/Sep/2024:18:17:00] ENGINE Error in HTTPServer.serve 2024-09-10T18:17:01.735 INFO:journalctl@ceph.mon.a.smithi115.stdout: Traceback (most recent call last): 2024-09-10T18:17:01.735 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:17:01.736 INFO:journalctl@ceph.mon.a.smithi115.stdout: self._connections.run(self.expiration_interval) 2024-09-10T18:17:01.736 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:17:01.736 INFO:journalctl@ceph.mon.a.smithi115.stdout: self._run(expiration_interval) 2024-09-10T18:17:01.736 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:17:01.736 INFO:journalctl@ceph.mon.a.smithi115.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:17:01.736 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:17:01.736 INFO:journalctl@ceph.mon.a.smithi115.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:17:01.736 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:17:01.736 INFO:journalctl@ceph.mon.a.smithi115.stdout: s = self.context.wrap_socket( 2024-09-10T18:17:01.736 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:17:01.736 INFO:journalctl@ceph.mon.a.smithi115.stdout: return self.sslsocket_class._create( 2024-09-10T18:17:01.736 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:17:01.736 INFO:journalctl@ceph.mon.a.smithi115.stdout: self.do_handshake() 2024-09-10T18:17:01.736 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:17:01.737 INFO:journalctl@ceph.mon.a.smithi115.stdout: self._sslobj.do_handshake() 2024-09-10T18:17:01.737 INFO:journalctl@ceph.mon.a.smithi115.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:17:01.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:01 smithi115 ceph-mon[29603]: from='mgr.14118 172.21.15.115:0/680636684' entity='mgr.y' 2024-09-10T18:17:01.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:01 smithi115 ceph-mon[29603]: from='mgr.14118 172.21.15.115:0/680636684' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:17:01.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:01 smithi115 ceph-mon[29603]: from='mgr.14118 172.21.15.115:0/680636684' entity='mgr.y' 2024-09-10T18:17:01.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:01 smithi115 ceph-mon[29603]: from='mgr.14118 172.21.15.115:0/680636684' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:17:02.947 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:02 smithi115 ceph-mon[29603]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:17:02.947 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:02 smithi115 ceph-mon[29603]: mgrmap e8: y(active, since 2s) 2024-09-10T18:17:03.208 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-10T18:17:03.208 INFO:teuthology.orchestra.run.smithi115.stdout:Generating ssh key... 2024-09-10T18:17:03.918 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:03 smithi115 ceph-mon[29603]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:17:04.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:03 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: Generating public/private rsa key pair. 2024-09-10T18:17:04.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:03 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: Your identification has been saved in /tmp/tmp_7n6ohp3/key 2024-09-10T18:17:04.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:03 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: Your public key has been saved in /tmp/tmp_7n6ohp3/key.pub 2024-09-10T18:17:04.236 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:03 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: The key fingerprint is: 2024-09-10T18:17:04.236 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:03 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: SHA256:8XlIZ+21LrN9CfdYdX7Ba/MLzvW+6FSqTkFcHTZevto ceph-a600e180-6fa0-11ef-bcea-c7b262605968 2024-09-10T18:17:04.236 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:03 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: The key's randomart image is: 2024-09-10T18:17:04.236 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:03 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: +---[RSA 3072]----+ 2024-09-10T18:17:04.236 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:03 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: | ..+o| 2024-09-10T18:17:04.236 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:03 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: | . ..ooo| 2024-09-10T18:17:04.236 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:03 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: | . .oo o.o| 2024-09-10T18:17:04.236 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:03 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: | +.= . +=| 2024-09-10T18:17:04.236 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:03 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: | S +.. .==| 2024-09-10T18:17:04.236 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:03 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: | ...*=+| 2024-09-10T18:17:04.236 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:03 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: | . B+E*| 2024-09-10T18:17:04.236 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:03 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: | . = X.=| 2024-09-10T18:17:04.236 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:03 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: | .o.* =*| 2024-09-10T18:17:04.236 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:03 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: +----[SHA256]-----+ 2024-09-10T18:17:04.780 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:04 smithi115 ceph-mon[29603]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:17:04.781 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:04 smithi115 ceph-mon[29603]: Generating ssh key... 2024-09-10T18:17:04.781 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:04 smithi115 ceph-mon[29603]: from='mgr.14118 172.21.15.115:0/680636684' entity='mgr.y' 2024-09-10T18:17:04.781 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:04 smithi115 ceph-mon[29603]: from='mgr.14118 172.21.15.115:0/680636684' entity='mgr.y' 2024-09-10T18:17:05.703 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyVcc4YA3CZFjx6OhqKb3x6xiPXcuFYi55IFIQxw08VbeH9fGcwzWGv4XYf20noD/9NwwLQVJlAf96AR21IlrSDyJ0K7KxSiJwvEIeIm13UI6og1lvKPG7lEjJEz7YruweY7pYnfCLTjkWgkDiO8GIfZ23LVgNdjURxizECxsEVrosnJl5rAn5W/wWH/mmtE9qBP0meaaxK8JIGzQXJsrvF4eCPr0iKH8/mmM3vgB5xQQUQXJlU9tLY4D+bQgxm9qYDX7Uqw24jEh3tWH9DJQrULWzjjnSOw/Ardf5smRwlXM4SabS1dl/gS6u6LFcZUb5fCHBr8ImOYbs0prDIbM3qTP7wOnTtAzix8n9Wv6VYBcc0N2nXcRl+yGGUNnRtdF2eCCG8pwGWgMW4HlN4+a8v08XeX4Qtst3yEjbgSWtifnv12xSqBiwI5cV3BNzu3exAkDhPNzEYywpw+LW9vM5wds39xD9TrvVckygrTeWRNDrpt9v2HMsMQUwkpytzQ0= ceph-a600e180-6fa0-11ef-bcea-c7b262605968 2024-09-10T18:17:05.703 INFO:teuthology.orchestra.run.smithi115.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-10T18:17:05.704 INFO:teuthology.orchestra.run.smithi115.stdout:Adding key to root@localhost authorized_keys... 2024-09-10T18:17:05.709 INFO:teuthology.orchestra.run.smithi115.stdout:Adding host smithi115... 2024-09-10T18:17:05.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:05 smithi115 ceph-mon[29603]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:17:07.092 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:06 smithi115 ceph-mon[29603]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi115", "addr": "172.21.15.115", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:17:08.012 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:07 smithi115 ceph-mon[29603]: Deploying cephadm binary to smithi115 2024-09-10T18:17:09.580 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout Added host 'smithi115' with addr '172.21.15.115' 2024-09-10T18:17:09.580 INFO:teuthology.orchestra.run.smithi115.stdout:Deploying unmanaged mon service... 2024-09-10T18:17:10.228 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:09 smithi115 ceph-mon[29603]: from='mgr.14118 172.21.15.115:0/680636684' entity='mgr.y' 2024-09-10T18:17:10.228 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:09 smithi115 ceph-mon[29603]: Added host smithi115 2024-09-10T18:17:10.228 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:09 smithi115 ceph-mon[29603]: from='mgr.14118 172.21.15.115:0/680636684' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:17:11.024 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-10T18:17:11.024 INFO:teuthology.orchestra.run.smithi115.stdout:Deploying unmanaged mgr service... 2024-09-10T18:17:11.566 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:11 smithi115 ceph-mon[29603]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:17:11.567 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:11 smithi115 ceph-mon[29603]: Saving service mon spec with placement count:5 2024-09-10T18:17:11.567 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:11 smithi115 ceph-mon[29603]: from='mgr.14118 172.21.15.115:0/680636684' entity='mgr.y' 2024-09-10T18:17:12.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:12 smithi115 ceph-mon[29603]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:17:12.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:12 smithi115 ceph-mon[29603]: Saving service mgr spec with placement count:2 2024-09-10T18:17:12.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:12 smithi115 ceph-mon[29603]: from='mgr.14118 172.21.15.115:0/680636684' entity='mgr.y' 2024-09-10T18:17:12.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:12 smithi115 ceph-mon[29603]: from='mgr.14118 172.21.15.115:0/680636684' entity='mgr.y' 2024-09-10T18:17:12.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:12 smithi115 ceph-mon[29603]: from='mgr.14118 172.21.15.115:0/680636684' entity='mgr.y' 2024-09-10T18:17:12.893 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-10T18:17:14.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:14 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/655611118' entity='client.admin' 2024-09-10T18:17:14.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:14 smithi115 ceph-mon[29603]: from='mgr.14118 172.21.15.115:0/680636684' entity='mgr.y' 2024-09-10T18:17:15.884 INFO:teuthology.orchestra.run.smithi115.stdout:Enabling the dashboard module... 2024-09-10T18:17:16.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:16 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/3527141156' entity='client.admin' 2024-09-10T18:17:17.760 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:17 smithi115 ceph-mon[29603]: from='mgr.14118 172.21.15.115:0/680636684' entity='mgr.y' 2024-09-10T18:17:17.761 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:17 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/4239343239' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-10T18:17:17.761 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:17 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ignoring --setuser ceph since I am not root 2024-09-10T18:17:17.761 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:17 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ignoring --setgroup ceph since I am not root 2024-09-10T18:17:17.761 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:17 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:17:17.639+0000 7f9b8c6adec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:17:17.761 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:17 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:17:17.693+0000 7f9b8c6adec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:17:17.761 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:17 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:17:17.758+0000 7f9b8c6adec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:17:18.023 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:17 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:17:17.882+0000 7f9b8c6adec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:17:18.023 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:17:18.020+0000 7f9b8c6adec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:17:18.543 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:18 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/4239343239' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-10T18:17:18.543 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:18 smithi115 ceph-mon[29603]: mgrmap e9: y(active, since 18s) 2024-09-10T18:17:18.544 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:17:18.282+0000 7f9b8c6adec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:17:18.544 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:17:18.342+0000 7f9b8c6adec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:17:18.544 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:17:18.507+0000 7f9b8c6adec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:17:19.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:19 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/3031651371' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T18:17:19.735 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:19 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:17:19.422+0000 7f9b8c6adec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:17:19.735 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:19 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:17:19.489+0000 7f9b8c6adec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:17:20.021 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:17:20.021 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-10T18:17:20.021 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T18:17:20.022 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-10T18:17:20.022 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T18:17:20.022 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:17:20.022 INFO:teuthology.orchestra.run.smithi115.stdout:Waiting for the mgr to restart... 2024-09-10T18:17:20.022 INFO:teuthology.orchestra.run.smithi115.stdout:Waiting for mgr epoch 9... 2024-09-10T18:17:20.404 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:20 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:17:20.218+0000 7f9b8c6adec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:17:20.404 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:20 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:17:20.337+0000 7f9b8c6adec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:17:20.655 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:20 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:17:20.463+0000 7f9b8c6adec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:17:20.655 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:20 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:17:20.525+0000 7f9b8c6adec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:17:20.655 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:20 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:17:20.652+0000 7f9b8c6adec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:17:20.913 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:20 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:17:20.719+0000 7f9b8c6adec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:17:20.913 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:20 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:17:20.776+0000 7f9b8c6adec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:17:20.913 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:20 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:17:20.910+0000 7f9b8c6adec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:17:21.178 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:20 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:17:20.974+0000 7f9b8c6adec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:17:21.178 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:21 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:17:21.038+0000 7f9b8c6adec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:17:21.178 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:21 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:17:21.118+0000 7f9b8c6adec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:17:21.178 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:21 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:17:21.175+0000 7f9b8c6adec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:17:21.484 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:21 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:17:21.232+0000 7f9b8c6adec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:17:22.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:21 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:17:21.974+0000 7f9b8c6adec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:17:22.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:22 smithi115 ceph-mon[29603]: Active manager daemon y restarted 2024-09-10T18:17:22.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:22 smithi115 ceph-mon[29603]: Activating manager daemon y 2024-09-10T18:17:22.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:22 smithi115 ceph-mon[29603]: osdmap e3: 0 total, 0 up, 0 in 2024-09-10T18:17:22.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:22 smithi115 ceph-mon[29603]: mgrmap e10: y(active, starting, since 0.00311676s) 2024-09-10T18:17:22.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:22 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:17:22.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:22 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:17:22.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:22 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:17:22.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:22 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:17:22.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:22 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:17:22.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:22 smithi115 ceph-mon[29603]: Manager daemon y is now available 2024-09-10T18:17:22.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:22 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:17:22.238 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:22 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:17:22.238 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:22 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:17:23.485 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:23 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:17:23.442+0000 7f9b3d20a640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:18:17:23] ENGINE Error in HTTPServer.serve 2024-09-10T18:17:23.485 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:23 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: Traceback (most recent call last): 2024-09-10T18:17:23.485 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:23 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:17:23.485 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:23 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: self._connections.run(self.expiration_interval) 2024-09-10T18:17:23.486 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:23 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:17:23.486 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:23 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: self._run(expiration_interval) 2024-09-10T18:17:23.486 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:23 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:17:23.486 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:23 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:17:23.486 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:23 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:17:23.486 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:23 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:17:23.486 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:23 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:17:23.486 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:23 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: s = self.context.wrap_socket( 2024-09-10T18:17:23.486 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:23 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:17:23.486 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:23 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: return self.sslsocket_class._create( 2024-09-10T18:17:23.486 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:23 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:17:23.486 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:23 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: self.do_handshake() 2024-09-10T18:17:23.487 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:23 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:17:23.487 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:23 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: self._sslobj.do_handshake() 2024-09-10T18:17:23.487 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:23 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:17:23.487 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:17:23 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:17:23.839 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:17:23.839 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-10T18:17:23.839 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T18:17:23.840 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:17:23.840 INFO:teuthology.orchestra.run.smithi115.stdout:mgr epoch 9 is available 2024-09-10T18:17:23.840 INFO:teuthology.orchestra.run.smithi115.stdout:Generating a dashboard self-signed certificate... 2024-09-10T18:17:24.096 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:23 smithi115 ceph-mon[29603]: mgrmap e11: y(active, since 1.00532s) 2024-09-10T18:17:24.096 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:23 smithi115 ceph-mon[29603]: [10/Sep/2024:18:17:23] ENGINE Bus STARTING 2024-09-10T18:17:24.096 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:23 smithi115 ceph-mon[29603]: [10/Sep/2024:18:17:23] ENGINE Serving on https://172.21.15.115:7150 2024-09-10T18:17:24.096 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:23 smithi115 ceph-mon[29603]: [10/Sep/2024:18:17:23] ENGINE Bus STARTED 2024-09-10T18:17:24.096 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:23 smithi115 ceph-mon[29603]: [10/Sep/2024:18:17:23] ENGINE Error in HTTPServer.serve 2024-09-10T18:17:24.097 INFO:journalctl@ceph.mon.a.smithi115.stdout: Traceback (most recent call last): 2024-09-10T18:17:24.097 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:17:24.097 INFO:journalctl@ceph.mon.a.smithi115.stdout: self._connections.run(self.expiration_interval) 2024-09-10T18:17:24.097 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:17:24.097 INFO:journalctl@ceph.mon.a.smithi115.stdout: self._run(expiration_interval) 2024-09-10T18:17:24.097 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:17:24.097 INFO:journalctl@ceph.mon.a.smithi115.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:17:24.097 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:17:24.097 INFO:journalctl@ceph.mon.a.smithi115.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:17:24.097 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:17:24.097 INFO:journalctl@ceph.mon.a.smithi115.stdout: s = self.context.wrap_socket( 2024-09-10T18:17:24.097 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:17:24.097 INFO:journalctl@ceph.mon.a.smithi115.stdout: return self.sslsocket_class._create( 2024-09-10T18:17:24.098 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:17:24.098 INFO:journalctl@ceph.mon.a.smithi115.stdout: self.do_handshake() 2024-09-10T18:17:24.098 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:17:24.098 INFO:journalctl@ceph.mon.a.smithi115.stdout: self._sslobj.do_handshake() 2024-09-10T18:17:24.098 INFO:journalctl@ceph.mon.a.smithi115.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:17:24.098 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:23 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:17:24.098 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:23 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:17:25.153 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-10T18:17:25.153 INFO:teuthology.orchestra.run.smithi115.stdout:Creating initial admin user... 2024-09-10T18:17:25.420 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:25 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:17:25.420 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:25 smithi115 ceph-mon[29603]: mgrmap e12: y(active, since 2s) 2024-09-10T18:17:25.420 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:25 smithi115 ceph-mon[29603]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:17:25.420 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:25 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:17:25.420 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:25 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:17:27.338 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:27 smithi115 ceph-mon[29603]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:17:27.339 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:27 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:17:27.341 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$g9Q3OR9UxlKV5Fo0e3jTbuHuvxH9siJOLVdcOOmnbschGSWeKqwcW", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1725992246, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-10T18:17:27.341 INFO:teuthology.orchestra.run.smithi115.stdout:Fetching dashboard port number... 2024-09-10T18:17:28.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:28 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1448013133' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-10T18:17:28.564 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 8443 2024-09-10T18:17:28.575 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T18:17:28.575 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout disabled 2024-09-10T18:17:28.583 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T18:17:28.583 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout inactive 2024-09-10T18:17:28.583 INFO:teuthology.orchestra.run.smithi115.stdout:firewalld.service is not enabled 2024-09-10T18:17:28.583 INFO:teuthology.orchestra.run.smithi115.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-10T18:17:28.585 INFO:teuthology.orchestra.run.smithi115.stdout:Ceph Dashboard is now available at: 2024-09-10T18:17:28.585 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:17:28.585 INFO:teuthology.orchestra.run.smithi115.stdout: URL: https://smithi115.front.sepia.ceph.com:8443/ 2024-09-10T18:17:28.585 INFO:teuthology.orchestra.run.smithi115.stdout: User: admin 2024-09-10T18:17:28.585 INFO:teuthology.orchestra.run.smithi115.stdout: Password: ithd1bjwpw 2024-09-10T18:17:28.585 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:17:28.586 INFO:teuthology.orchestra.run.smithi115.stdout:Saving cluster configuration to /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config directory 2024-09-10T18:17:28.586 INFO:teuthology.orchestra.run.smithi115.stdout:Enabling autotune for osd_memory_target 2024-09-10T18:17:29.073 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:29 smithi115 ceph-mon[29603]: mgrmap e13: y(active, since 6s) 2024-09-10T18:17:30.977 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-10T18:17:30.977 INFO:teuthology.orchestra.run.smithi115.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-10T18:17:30.978 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:17:30.978 INFO:teuthology.orchestra.run.smithi115.stdout: sudo /sbin/cephadm shell --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:17:30.978 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:17:30.978 INFO:teuthology.orchestra.run.smithi115.stdout:Or, if you are only running a single cluster on this host: 2024-09-10T18:17:30.978 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:17:30.978 INFO:teuthology.orchestra.run.smithi115.stdout: sudo /sbin/cephadm shell 2024-09-10T18:17:30.978 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:17:30.978 INFO:teuthology.orchestra.run.smithi115.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-10T18:17:30.978 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:17:30.978 INFO:teuthology.orchestra.run.smithi115.stdout: ceph telemetry on 2024-09-10T18:17:30.978 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:17:30.978 INFO:teuthology.orchestra.run.smithi115.stdout:For more information see: 2024-09-10T18:17:30.978 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:17:30.979 INFO:teuthology.orchestra.run.smithi115.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-09-10T18:17:30.979 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:17:30.979 INFO:teuthology.orchestra.run.smithi115.stdout:Bootstrap complete. 2024-09-10T18:17:31.013 INFO:tasks.cephadm:Fetching config... 2024-09-10T18:17:31.013 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T18:17:31.013 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-10T18:17:31.029 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-10T18:17:31.029 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T18:17:31.029 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-10T18:17:31.085 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-10T18:17:31.086 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T18:17:31.086 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-10T18:17:31.150 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-10T18:17:31.150 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T18:17:31.151 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-10T18:17:31.206 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-10T18:17:31.206 DEBUG:teuthology.orchestra.run.smithi115:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyVcc4YA3CZFjx6OhqKb3x6xiPXcuFYi55IFIQxw08VbeH9fGcwzWGv4XYf20noD/9NwwLQVJlAf96AR21IlrSDyJ0K7KxSiJwvEIeIm13UI6og1lvKPG7lEjJEz7YruweY7pYnfCLTjkWgkDiO8GIfZ23LVgNdjURxizECxsEVrosnJl5rAn5W/wWH/mmtE9qBP0meaaxK8JIGzQXJsrvF4eCPr0iKH8/mmM3vgB5xQQUQXJlU9tLY4D+bQgxm9qYDX7Uqw24jEh3tWH9DJQrULWzjjnSOw/Ardf5smRwlXM4SabS1dl/gS6u6LFcZUb5fCHBr8ImOYbs0prDIbM3qTP7wOnTtAzix8n9Wv6VYBcc0N2nXcRl+yGGUNnRtdF2eCCG8pwGWgMW4HlN4+a8v08XeX4Qtst3yEjbgSWtifnv12xSqBiwI5cV3BNzu3exAkDhPNzEYywpw+LW9vM5wds39xD9TrvVckygrTeWRNDrpt9v2HMsMQUwkpytzQ0= ceph-a600e180-6fa0-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T18:17:31.280 INFO:teuthology.orchestra.run.smithi115.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyVcc4YA3CZFjx6OhqKb3x6xiPXcuFYi55IFIQxw08VbeH9fGcwzWGv4XYf20noD/9NwwLQVJlAf96AR21IlrSDyJ0K7KxSiJwvEIeIm13UI6og1lvKPG7lEjJEz7YruweY7pYnfCLTjkWgkDiO8GIfZ23LVgNdjURxizECxsEVrosnJl5rAn5W/wWH/mmtE9qBP0meaaxK8JIGzQXJsrvF4eCPr0iKH8/mmM3vgB5xQQUQXJlU9tLY4D+bQgxm9qYDX7Uqw24jEh3tWH9DJQrULWzjjnSOw/Ardf5smRwlXM4SabS1dl/gS6u6LFcZUb5fCHBr8ImOYbs0prDIbM3qTP7wOnTtAzix8n9Wv6VYBcc0N2nXcRl+yGGUNnRtdF2eCCG8pwGWgMW4HlN4+a8v08XeX4Qtst3yEjbgSWtifnv12xSqBiwI5cV3BNzu3exAkDhPNzEYywpw+LW9vM5wds39xD9TrvVckygrTeWRNDrpt9v2HMsMQUwkpytzQ0= ceph-a600e180-6fa0-11ef-bcea-c7b262605968 2024-09-10T18:17:31.291 DEBUG:teuthology.orchestra.run.smithi150:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyVcc4YA3CZFjx6OhqKb3x6xiPXcuFYi55IFIQxw08VbeH9fGcwzWGv4XYf20noD/9NwwLQVJlAf96AR21IlrSDyJ0K7KxSiJwvEIeIm13UI6og1lvKPG7lEjJEz7YruweY7pYnfCLTjkWgkDiO8GIfZ23LVgNdjURxizECxsEVrosnJl5rAn5W/wWH/mmtE9qBP0meaaxK8JIGzQXJsrvF4eCPr0iKH8/mmM3vgB5xQQUQXJlU9tLY4D+bQgxm9qYDX7Uqw24jEh3tWH9DJQrULWzjjnSOw/Ardf5smRwlXM4SabS1dl/gS6u6LFcZUb5fCHBr8ImOYbs0prDIbM3qTP7wOnTtAzix8n9Wv6VYBcc0N2nXcRl+yGGUNnRtdF2eCCG8pwGWgMW4HlN4+a8v08XeX4Qtst3yEjbgSWtifnv12xSqBiwI5cV3BNzu3exAkDhPNzEYywpw+LW9vM5wds39xD9TrvVckygrTeWRNDrpt9v2HMsMQUwkpytzQ0= ceph-a600e180-6fa0-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T18:17:31.397 INFO:teuthology.orchestra.run.smithi150.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyVcc4YA3CZFjx6OhqKb3x6xiPXcuFYi55IFIQxw08VbeH9fGcwzWGv4XYf20noD/9NwwLQVJlAf96AR21IlrSDyJ0K7KxSiJwvEIeIm13UI6og1lvKPG7lEjJEz7YruweY7pYnfCLTjkWgkDiO8GIfZ23LVgNdjURxizECxsEVrosnJl5rAn5W/wWH/mmtE9qBP0meaaxK8JIGzQXJsrvF4eCPr0iKH8/mmM3vgB5xQQUQXJlU9tLY4D+bQgxm9qYDX7Uqw24jEh3tWH9DJQrULWzjjnSOw/Ardf5smRwlXM4SabS1dl/gS6u6LFcZUb5fCHBr8ImOYbs0prDIbM3qTP7wOnTtAzix8n9Wv6VYBcc0N2nXcRl+yGGUNnRtdF2eCCG8pwGWgMW4HlN4+a8v08XeX4Qtst3yEjbgSWtifnv12xSqBiwI5cV3BNzu3exAkDhPNzEYywpw+LW9vM5wds39xD9TrvVckygrTeWRNDrpt9v2HMsMQUwkpytzQ0= ceph-a600e180-6fa0-11ef-bcea-c7b262605968 2024-09-10T18:17:31.411 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-10T18:17:31.527 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:31 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/630652322' entity='client.admin' 2024-09-10T18:17:31.658 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:17:35.134 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-10T18:17:35.134 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-10T18:17:35.477 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:35 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/3001457992' entity='client.admin' 2024-09-10T18:17:35.556 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:17:36.666 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:36 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:17:36.667 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:36 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:17:36.667 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:36 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:17:36.667 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:36 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:17:36.667 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:36 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:17:36.667 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:36 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:17:38.093 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi150 2024-09-10T18:17:38.093 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-10T18:17:38.093 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/etc/ceph/ceph.conf 2024-09-10T18:17:38.112 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-10T18:17:38.112 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:17:38.171 INFO:tasks.cephadm:Adding host smithi150 to orchestrator... 2024-09-10T18:17:38.171 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph orch host add smithi150 2024-09-10T18:17:38.408 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:17:38.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:38 smithi115 ceph-mon[29603]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:17:38.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:38 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:17:38.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:38 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:17:38.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:38 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:38.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:38 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:17:38.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:38 smithi115 ceph-mon[29603]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-10T18:17:38.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:38 smithi115 ceph-mon[29603]: Updating smithi115:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:17:38.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:38 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:17:38.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:38 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:17:38.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:38 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:17:39.499 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:39 smithi115 ceph-mon[29603]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:17:39.499 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:39 smithi115 ceph-mon[29603]: Updating smithi115:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:17:40.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:40 smithi115 ceph-mon[29603]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi150", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:17:41.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:41 smithi115 ceph-mon[29603]: Deploying cephadm binary to smithi150 2024-09-10T18:17:42.869 INFO:teuthology.orchestra.run.smithi115.stdout:Added host 'smithi150' with addr '172.21.15.150' 2024-09-10T18:17:43.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:42 smithi115 ceph-mon[29603]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:17:43.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:42 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:17:43.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:42 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:17:43.538 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T18:17:43.776 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:17:44.234 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:44 smithi115 ceph-mon[29603]: Added host smithi150 2024-09-10T18:17:44.234 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:44 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:17:44.234 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:44 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:17:45.231 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:45 smithi115 ceph-mon[29603]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:17:45.231 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:45 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:17:45.475 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:17:45.475 INFO:teuthology.orchestra.run.smithi115.stdout:[{"addr": "172.21.15.115", "hostname": "smithi115", "labels": [], "status": ""}, {"addr": "172.21.15.150", "hostname": "smithi150", "labels": [], "status": ""}] 2024-09-10T18:17:46.127 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-10T18:17:46.128 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-10T18:17:46.366 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:17:46.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:46 smithi115 ceph-mon[29603]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:17:47.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:47 smithi115 ceph-mon[29603]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:17:47.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:47 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:17:47.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:47 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:17:47.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:47 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:17:47.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:47 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:17:47.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:47 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi150", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:17:47.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:47 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:47.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:47 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:17:48.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:48 smithi115 ceph-mon[29603]: Updating smithi150:/etc/ceph/ceph.conf 2024-09-10T18:17:48.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:48 smithi115 ceph-mon[29603]: Updating smithi150:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:17:48.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:48 smithi115 ceph-mon[29603]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:17:48.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:48 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:17:48.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:48 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:17:48.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:48 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:17:49.231 INFO:teuthology.orchestra.run.smithi115.stderr:adjusted tunables profile to default 2024-09-10T18:17:49.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:49 smithi115 ceph-mon[29603]: Updating smithi150:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:17:49.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:49 smithi115 ceph-mon[29603]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:17:49.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:49 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/3670855908' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T18:17:49.877 INFO:tasks.cephadm:Adding mon.a on smithi115 2024-09-10T18:17:49.877 INFO:tasks.cephadm:Adding mon.c on smithi115 2024-09-10T18:17:49.877 INFO:tasks.cephadm:Adding mon.b on smithi150 2024-09-10T18:17:49.877 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi115:172.21.15.115=a;smithi115:[v2:172.21.15.115:3301,v1:172.21.15.115:6790]=c;smithi150:172.21.15.150=b' 2024-09-10T18:17:50.123 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:17:50.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:50 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/3670855908' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T18:17:50.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:50 smithi115 ceph-mon[29603]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:17:51.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:51 smithi115 ceph-mon[29603]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:17:51.757 INFO:teuthology.orchestra.run.smithi150.stdout:Scheduled mon update... 2024-09-10T18:17:52.351 DEBUG:teuthology.orchestra.run.smithi115:mon.c> sudo journalctl -f -n 0 -u ceph-a600e180-6fa0-11ef-bcea-c7b262605968@mon.c.service 2024-09-10T18:17:52.355 DEBUG:teuthology.orchestra.run.smithi150:mon.b> sudo journalctl -f -n 0 -u ceph-a600e180-6fa0-11ef-bcea-c7b262605968@mon.b.service 2024-09-10T18:17:52.357 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T18:17:52.357 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T18:17:53.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:52 smithi115 ceph-mon[29603]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi115:172.21.15.115=a;smithi115:[v2:172.21.15.115:3301,v1:172.21.15.115:6790]=c;smithi150:172.21.15.150=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:17:53.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:52 smithi115 ceph-mon[29603]: Saving service mon spec with placement smithi115:172.21.15.115=a;smithi115:[v2:172.21.15.115:3301,v1:172.21.15.115:6790]=c;smithi150:172.21.15.150=b;count:3 2024-09-10T18:17:53.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:52 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:17:53.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:52 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:17:53.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:52 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:53.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:52 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:17:53.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:52 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:17:53.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:52 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:17:53.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:52 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:53.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:52 smithi115 ceph-mon[29603]: Deploying daemon mon.b on smithi150 2024-09-10T18:17:53.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:52 smithi115 ceph-mon[29603]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:17:53.889 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:17:55.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:55 smithi150 systemd[1]: Starting Ceph mon.b for a600e180-6fa0-11ef-bcea-c7b262605968... 2024-09-10T18:17:55.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:17:55 smithi115 ceph-mon[29603]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:17:56.210 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:17:56.210 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":1,"fsid":"a600e180-6fa0-11ef-bcea-c7b262605968","modified":"2024-09-10T18:16:23.954068Z","created":"2024-09-10T18:16:23.954068Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3300","nonce":0},{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T18:17:56.210 INFO:teuthology.orchestra.run.smithi150.stderr:dumped monmap epoch 1 2024-09-10T18:17:56.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 podman[32701]: 2024-09-10 18:17:56.081994327 +0000 UTC m=+0.109055209 container create bb0215e57dbdd1ee204189c64c28d1f943bebb295ef3a8aa0c38aaea2365a4ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-b, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-10T18:17:56.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 podman[32701]: 2024-09-10 18:17:55.994122237 +0000 UTC m=+0.021183132 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:17:56.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 podman[32701]: 2024-09-10 18:17:56.284632653 +0000 UTC m=+0.311693534 container init bb0215e57dbdd1ee204189c64c28d1f943bebb295ef3a8aa0c38aaea2365a4ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-b, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-10T18:17:56.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 podman[32701]: 2024-09-10 18:17:56.289092229 +0000 UTC m=+0.316153111 container start bb0215e57dbdd1ee204189c64c28d1f943bebb295ef3a8aa0c38aaea2365a4ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=) 2024-09-10T18:17:56.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T18:17:56.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable), process ceph-mon, pid 7 2024-09-10T18:17:56.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: pidfile_write: ignore empty --pid-file 2024-09-10T18:17:56.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: load: jerasure load: lrc 2024-09-10T18:17:56.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: RocksDB version: 6.15.5 2024-09-10T18:17:56.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-10T18:17:56.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Compile date Sep 9 2024 2024-09-10T18:17:56.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: DB SUMMARY 2024-09-10T18:17:56.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: DB Session ID: X5T36WMQU4XX93EQ8WTT 2024-09-10T18:17:56.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: CURRENT file: CURRENT 2024-09-10T18:17:56.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T18:17:56.557 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-09-10T18:17:56.557 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-10T18:17:56.557 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 511 ; 2024-09-10T18:17:56.557 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.error_if_exists: 0 2024-09-10T18:17:56.557 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.create_if_missing: 0 2024-09-10T18:17:56.557 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.paranoid_checks: 1 2024-09-10T18:17:56.557 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T18:17:56.557 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.env: 0x55f29ad34440 2024-09-10T18:17:56.557 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.fs: Posix File System 2024-09-10T18:17:56.557 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.info_log: 0x55f29c8c4ec0 2024-09-10T18:17:56.557 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T18:17:56.557 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.statistics: (nil) 2024-09-10T18:17:56.559 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.use_fsync: 0 2024-09-10T18:17:56.559 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.max_log_file_size: 0 2024-09-10T18:17:56.559 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T18:17:56.559 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T18:17:56.559 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T18:17:56.559 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T18:17:56.559 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.allow_fallocate: 1 2024-09-10T18:17:56.560 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T18:17:56.560 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T18:17:56.560 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.use_direct_reads: 0 2024-09-10T18:17:56.560 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T18:17:56.560 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T18:17:56.560 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.db_log_dir: 2024-09-10T18:17:56.560 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-b/store.db 2024-09-10T18:17:56.560 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T18:17:56.560 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T18:17:56.560 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T18:17:56.560 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T18:17:56.561 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T18:17:56.561 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T18:17:56.561 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T18:17:56.561 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T18:17:56.561 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.write_buffer_manager: 0x55f29c982780 2024-09-10T18:17:56.561 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T18:17:56.561 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-10T18:17:56.561 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T18:17:56.561 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T18:17:56.561 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T18:17:56.561 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T18:17:56.561 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T18:17:56.562 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T18:17:56.562 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T18:17:56.562 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.unordered_write: 0 2024-09-10T18:17:56.562 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T18:17:56.562 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T18:17:56.562 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T18:17:56.562 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T18:17:56.562 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.row_cache: None 2024-09-10T18:17:56.562 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.wal_filter: None 2024-09-10T18:17:56.562 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T18:17:56.562 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T18:17:56.562 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.preserve_deletes: 0 2024-09-10T18:17:56.562 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.two_write_queues: 0 2024-09-10T18:17:56.563 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T18:17:56.563 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.atomic_flush: 0 2024-09-10T18:17:56.563 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T18:17:56.563 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T18:17:56.563 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T18:17:56.563 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.log_readahead_size: 0 2024-09-10T18:17:56.563 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T18:17:56.563 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T18:17:56.563 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T18:17:56.563 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T18:17:56.563 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T18:17:56.563 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T18:17:56.563 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.max_background_jobs: 2 2024-09-10T18:17:56.564 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.max_background_compactions: -1 2024-09-10T18:17:56.564 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.max_subcompactions: 1 2024-09-10T18:17:56.564 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T18:17:56.564 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T18:17:56.564 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T18:17:56.564 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T18:17:56.564 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T18:17:56.564 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T18:17:56.564 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T18:17:56.564 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T18:17:56.564 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.max_open_files: -1 2024-09-10T18:17:56.564 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T18:17:56.564 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T18:17:56.565 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T18:17:56.565 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T18:17:56.565 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.max_background_flushes: -1 2024-09-10T18:17:56.565 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Compression algorithms supported: 2024-09-10T18:17:56.565 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: kZSTD supported: 0 2024-09-10T18:17:56.565 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: kXpressCompression supported: 0 2024-09-10T18:17:56.565 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: kBZip2Compression supported: 0 2024-09-10T18:17:56.565 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T18:17:56.565 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: kLZ4Compression supported: 1 2024-09-10T18:17:56.565 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: kZlibCompression supported: 1 2024-09-10T18:17:56.565 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T18:17:56.565 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: kSnappyCompression supported: 1 2024-09-10T18:17:56.566 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T18:17:56.566 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000003 2024-09-10T18:17:56.566 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-10T18:17:56.566 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T18:17:56.566 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.merge_operator: 2024-09-10T18:17:56.566 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.compaction_filter: None 2024-09-10T18:17:56.566 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T18:17:56.566 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T18:17:56.566 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T18:17:56.566 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T18:17:56.566 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f29c86ba38) 2024-09-10T18:17:56.566 INFO:journalctl@ceph.mon.b.smithi150.stdout: cache_index_and_filter_blocks: 1 2024-09-10T18:17:56.567 INFO:journalctl@ceph.mon.b.smithi150.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T18:17:56.567 INFO:journalctl@ceph.mon.b.smithi150.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T18:17:56.567 INFO:journalctl@ceph.mon.b.smithi150.stdout: pin_top_level_index_and_filter: 1 2024-09-10T18:17:56.567 INFO:journalctl@ceph.mon.b.smithi150.stdout: index_type: 0 2024-09-10T18:17:56.567 INFO:journalctl@ceph.mon.b.smithi150.stdout: data_block_index_type: 0 2024-09-10T18:17:56.567 INFO:journalctl@ceph.mon.b.smithi150.stdout: index_shortening: 1 2024-09-10T18:17:56.567 INFO:journalctl@ceph.mon.b.smithi150.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T18:17:56.567 INFO:journalctl@ceph.mon.b.smithi150.stdout: hash_index_allow_collision: 1 2024-09-10T18:17:56.567 INFO:journalctl@ceph.mon.b.smithi150.stdout: checksum: 1 2024-09-10T18:17:56.567 INFO:journalctl@ceph.mon.b.smithi150.stdout: no_block_cache: 0 2024-09-10T18:17:56.567 INFO:journalctl@ceph.mon.b.smithi150.stdout: block_cache: 0x55f29c8a1350 2024-09-10T18:17:56.567 INFO:journalctl@ceph.mon.b.smithi150.stdout: block_cache_name: BinnedLRUCache 2024-09-10T18:17:56.567 INFO:journalctl@ceph.mon.b.smithi150.stdout: block_cache_options: 2024-09-10T18:17:56.568 INFO:journalctl@ceph.mon.b.smithi150.stdout: capacity : 536870912 2024-09-10T18:17:56.568 INFO:journalctl@ceph.mon.b.smithi150.stdout: num_shard_bits : 4 2024-09-10T18:17:56.568 INFO:journalctl@ceph.mon.b.smithi150.stdout: strict_capacity_limit : 0 2024-09-10T18:17:56.568 INFO:journalctl@ceph.mon.b.smithi150.stdout: high_pri_pool_ratio: 0.000 2024-09-10T18:17:56.568 INFO:journalctl@ceph.mon.b.smithi150.stdout: block_cache_compressed: (nil) 2024-09-10T18:17:56.568 INFO:journalctl@ceph.mon.b.smithi150.stdout: persistent_cache: (nil) 2024-09-10T18:17:56.568 INFO:journalctl@ceph.mon.b.smithi150.stdout: block_size: 4096 2024-09-10T18:17:56.568 INFO:journalctl@ceph.mon.b.smithi150.stdout: block_size_deviation: 10 2024-09-10T18:17:56.568 INFO:journalctl@ceph.mon.b.smithi150.stdout: block_restart_interval: 16 2024-09-10T18:17:56.568 INFO:journalctl@ceph.mon.b.smithi150.stdout: index_block_restart_interval: 1 2024-09-10T18:17:56.568 INFO:journalctl@ceph.mon.b.smithi150.stdout: metadata_block_size: 4096 2024-09-10T18:17:56.568 INFO:journalctl@ceph.mon.b.smithi150.stdout: partition_filters: 0 2024-09-10T18:17:56.568 INFO:journalctl@ceph.mon.b.smithi150.stdout: use_delta_encoding: 1 2024-09-10T18:17:56.569 INFO:journalctl@ceph.mon.b.smithi150.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-10T18:17:56.569 INFO:journalctl@ceph.mon.b.smithi150.stdout: whole_key_filtering: 1 2024-09-10T18:17:56.569 INFO:journalctl@ceph.mon.b.smithi150.stdout: verify_compression: 0 2024-09-10T18:17:56.569 INFO:journalctl@ceph.mon.b.smithi150.stdout: read_amp_bytes_per_bit: 0 2024-09-10T18:17:56.569 INFO:journalctl@ceph.mon.b.smithi150.stdout: format_version: 4 2024-09-10T18:17:56.569 INFO:journalctl@ceph.mon.b.smithi150.stdout: enable_index_compression: 1 2024-09-10T18:17:56.569 INFO:journalctl@ceph.mon.b.smithi150.stdout: block_align: 0 2024-09-10T18:17:56.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T18:17:56.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T18:17:56.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.compression: NoCompression 2024-09-10T18:17:56.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T18:17:56.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T18:17:56.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T18:17:56.570 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.num_levels: 7 2024-09-10T18:17:56.570 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T18:17:56.570 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T18:17:56.570 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T18:17:56.570 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T18:17:56.570 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T18:17:56.570 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T18:17:56.570 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T18:17:56.570 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:17:56.570 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T18:17:56.570 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T18:17:56.570 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T18:17:56.570 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T18:17:56.571 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T18:17:56.571 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T18:17:56.571 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:17:56.571 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T18:17:56.571 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T18:17:56.571 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T18:17:56.571 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T18:17:56.571 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T18:17:56.571 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T18:17:56.571 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T18:17:56.571 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T18:17:56.571 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T18:17:56.571 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T18:17:56.572 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T18:17:56.572 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T18:17:56.572 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T18:17:56.572 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T18:17:56.572 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T18:17:56.572 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T18:17:56.572 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T18:17:56.572 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T18:17:56.572 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T18:17:56.572 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.arena_block_size: 4194304 2024-09-10T18:17:56.572 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T18:17:56.572 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T18:17:56.572 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-10T18:17:56.573 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T18:17:56.573 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T18:17:56.573 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T18:17:56.573 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T18:17:56.573 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T18:17:56.573 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T18:17:56.573 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T18:17:56.573 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T18:17:56.573 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T18:17:56.573 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T18:17:56.573 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T18:17:56.573 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T18:17:56.573 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.inplace_update_support: 0 2024-09-10T18:17:56.573 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T18:17:56.574 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T18:17:56.574 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T18:17:56.574 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T18:17:56.574 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.bloom_locality: 0 2024-09-10T18:17:56.574 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.max_successive_merges: 0 2024-09-10T18:17:56.574 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T18:17:56.574 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T18:17:56.574 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T18:17:56.574 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T18:17:56.574 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.ttl: 2592000 2024-09-10T18:17:56.574 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T18:17:56.574 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.enable_blob_files: false 2024-09-10T18:17:56.574 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.min_blob_size: 0 2024-09-10T18:17:56.575 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T18:17:56.575 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T18:17:56.575 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T18:17:56.575 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T18:17:56.575 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T18:17:56.575 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-09-10T18:17:56.575 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-09-10T18:17:56.575 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725992276313425, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-10T18:17:56.575 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-09-10T18:17:56.575 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-10T18:17:56.575 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725992276313739, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1571, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1725992276, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "c23b7e4e-3030-4f6e-8c3b-6d69241d06b6", "db_session_id": "X5T36WMQU4XX93EQ8WTT"}} 2024-09-10T18:17:56.575 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-09-10T18:17:56.575 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725992276314071, "job": 1, "event": "recovery_finished"} 2024-09-10T18:17:56.576 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T18:17:56.576 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55f29c8d4700 2024-09-10T18:17:56.576 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: DB pointer 0x55f29c8a4000 2024-09-10T18:17:56.576 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-10T18:17:56.576 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-10T18:17:56.576 INFO:journalctl@ceph.mon.b.smithi150.stdout: ** DB Stats ** 2024-09-10T18:17:56.576 INFO:journalctl@ceph.mon.b.smithi150.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:17:56.576 INFO:journalctl@ceph.mon.b.smithi150.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T18:17:56.576 INFO:journalctl@ceph.mon.b.smithi150.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T18:17:56.576 INFO:journalctl@ceph.mon.b.smithi150.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:17:56.576 INFO:journalctl@ceph.mon.b.smithi150.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T18:17:56.576 INFO:journalctl@ceph.mon.b.smithi150.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-10T18:17:56.576 INFO:journalctl@ceph.mon.b.smithi150.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:17:56.577 INFO:journalctl@ceph.mon.b.smithi150.stdout: 2024-09-10T18:17:56.577 INFO:journalctl@ceph.mon.b.smithi150.stdout: ** Compaction Stats [default] ** 2024-09-10T18:17:56.577 INFO:journalctl@ceph.mon.b.smithi150.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:17:56.577 INFO:journalctl@ceph.mon.b.smithi150.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:17:56.577 INFO:journalctl@ceph.mon.b.smithi150.stdout: L0 1/0 1.53 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.0 0.00 0.00 1 0.000 0 0 2024-09-10T18:17:56.577 INFO:journalctl@ceph.mon.b.smithi150.stdout: Sum 1/0 1.53 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.0 0.00 0.00 1 0.000 0 0 2024-09-10T18:17:56.577 INFO:journalctl@ceph.mon.b.smithi150.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.0 0.00 0.00 1 0.000 0 0 2024-09-10T18:17:56.577 INFO:journalctl@ceph.mon.b.smithi150.stdout: 2024-09-10T18:17:56.577 INFO:journalctl@ceph.mon.b.smithi150.stdout: ** Compaction Stats [default] ** 2024-09-10T18:17:56.577 INFO:journalctl@ceph.mon.b.smithi150.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:17:56.577 INFO:journalctl@ceph.mon.b.smithi150.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:17:56.578 INFO:journalctl@ceph.mon.b.smithi150.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.0 0.00 0.00 1 0.000 0 0 2024-09-10T18:17:56.578 INFO:journalctl@ceph.mon.b.smithi150.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:17:56.578 INFO:journalctl@ceph.mon.b.smithi150.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:17:56.578 INFO:journalctl@ceph.mon.b.smithi150.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:17:56.578 INFO:journalctl@ceph.mon.b.smithi150.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:17:56.578 INFO:journalctl@ceph.mon.b.smithi150.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:17:56.578 INFO:journalctl@ceph.mon.b.smithi150.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:17:56.578 INFO:journalctl@ceph.mon.b.smithi150.stdout: Cumulative compaction: 0.00 GB write, 0.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:17:56.578 INFO:journalctl@ceph.mon.b.smithi150.stdout: Interval compaction: 0.00 GB write, 0.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:17:56.578 INFO:journalctl@ceph.mon.b.smithi150.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:17:56.578 INFO:journalctl@ceph.mon.b.smithi150.stdout: 2024-09-10T18:17:56.578 INFO:journalctl@ceph.mon.b.smithi150.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:17:56.578 INFO:journalctl@ceph.mon.b.smithi150.stdout: 2024-09-10T18:17:56.579 INFO:journalctl@ceph.mon.b.smithi150.stdout: ** Compaction Stats [default] ** 2024-09-10T18:17:56.579 INFO:journalctl@ceph.mon.b.smithi150.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:17:56.579 INFO:journalctl@ceph.mon.b.smithi150.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:17:56.579 INFO:journalctl@ceph.mon.b.smithi150.stdout: L0 1/0 1.53 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.0 0.00 0.00 1 0.000 0 0 2024-09-10T18:17:56.579 INFO:journalctl@ceph.mon.b.smithi150.stdout: Sum 1/0 1.53 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.0 0.00 0.00 1 0.000 0 0 2024-09-10T18:17:56.579 INFO:journalctl@ceph.mon.b.smithi150.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-10T18:17:56.579 INFO:journalctl@ceph.mon.b.smithi150.stdout: 2024-09-10T18:17:56.579 INFO:journalctl@ceph.mon.b.smithi150.stdout: ** Compaction Stats [default] ** 2024-09-10T18:17:56.579 INFO:journalctl@ceph.mon.b.smithi150.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:17:56.579 INFO:journalctl@ceph.mon.b.smithi150.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:17:56.579 INFO:journalctl@ceph.mon.b.smithi150.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.0 0.00 0.00 1 0.000 0 0 2024-09-10T18:17:56.579 INFO:journalctl@ceph.mon.b.smithi150.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:17:56.579 INFO:journalctl@ceph.mon.b.smithi150.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:17:56.580 INFO:journalctl@ceph.mon.b.smithi150.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:17:56.580 INFO:journalctl@ceph.mon.b.smithi150.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:17:56.580 INFO:journalctl@ceph.mon.b.smithi150.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:17:56.580 INFO:journalctl@ceph.mon.b.smithi150.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:17:56.580 INFO:journalctl@ceph.mon.b.smithi150.stdout: Cumulative compaction: 0.00 GB write, 0.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:17:56.580 INFO:journalctl@ceph.mon.b.smithi150.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:17:56.580 INFO:journalctl@ceph.mon.b.smithi150.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:17:56.580 INFO:journalctl@ceph.mon.b.smithi150.stdout: 2024-09-10T18:17:56.580 INFO:journalctl@ceph.mon.b.smithi150.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:17:56.580 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-10T18:17:56.580 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: using public_addr v2:172.21.15.150:0/0 -> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] 2024-09-10T18:17:56.580 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: starting mon.b rank -1 at public addrs [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] at bind addrs [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid a600e180-6fa0-11ef-bcea-c7b262605968 2024-09-10T18:17:56.580 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: mon.b@-1(???) e0 preinit fsid a600e180-6fa0-11ef-bcea-c7b262605968 2024-09-10T18:17:56.581 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: mon.b@-1(synchronizing).mds e1 new map 2024-09-10T18:17:56.581 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-10T18:17:56.581 INFO:journalctl@ceph.mon.b.smithi150.stdout: e1 2024-09-10T18:17:56.581 INFO:journalctl@ceph.mon.b.smithi150.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T18:17:56.581 INFO:journalctl@ceph.mon.b.smithi150.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T18:17:56.581 INFO:journalctl@ceph.mon.b.smithi150.stdout: legacy client fscid: -1 2024-09-10T18:17:56.581 INFO:journalctl@ceph.mon.b.smithi150.stdout: 2024-09-10T18:17:56.581 INFO:journalctl@ceph.mon.b.smithi150.stdout: No filesystems configured 2024-09-10T18:17:56.581 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-10T18:17:56.581 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-10T18:17:56.581 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-10T18:17:56.581 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-10T18:17:56.581 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-10T18:17:56.582 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-10T18:17:56.582 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T18:17:56.582 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:17:56.582 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:17:56.582 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:17:56.582 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: Updating smithi150:/etc/ceph/ceph.conf 2024-09-10T18:17:56.582 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: Updating smithi150:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:17:56.582 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:17:56.582 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:17:56.582 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:17:56.582 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:17:56.582 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: Updating smithi150:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:17:56.582 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:17:56.583 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/3670855908' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T18:17:56.583 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/3670855908' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T18:17:56.583 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:17:56.583 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:17:56.583 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi115:172.21.15.115=a;smithi115:[v2:172.21.15.115:3301,v1:172.21.15.115:6790]=c;smithi150:172.21.15.150=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:17:56.583 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: Saving service mon spec with placement smithi115:172.21.15.115=a;smithi115:[v2:172.21.15.115:3301,v1:172.21.15.115:6790]=c;smithi150:172.21.15.150=b;count:3 2024-09-10T18:17:56.583 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:17:56.583 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:17:56.583 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:56.583 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:17:56.583 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:17:56.583 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:17:56.583 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:56.584 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: Deploying daemon mon.b on smithi150 2024-09-10T18:17:56.584 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:17:56.584 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:17:56.584 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 ceph-mon[32761]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-10T18:17:56.584 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 bash[32701]: bb0215e57dbdd1ee204189c64c28d1f943bebb295ef3a8aa0c38aaea2365a4ed 2024-09-10T18:17:56.584 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:17:56 smithi150 systemd[1]: Started Ceph mon.b for a600e180-6fa0-11ef-bcea-c7b262605968. 2024-09-10T18:17:57.988 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T18:17:57.988 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T18:17:58.251 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:18:01.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:01 smithi115 ceph-mon[29603]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:01.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:01 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:18:01.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:01 smithi115 ceph-mon[29603]: mon.a calling monitor election 2024-09-10T18:18:01.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:01 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:18:01.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:01 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:18:01.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:01 smithi115 ceph-mon[29603]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:01.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:01 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:18:01.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:01 smithi115 ceph-mon[29603]: mon.b calling monitor election 2024-09-10T18:18:01.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:01 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:18:01.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:01 smithi115 ceph-mon[29603]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:01.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:01 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:18:01.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:01 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:18:01.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:01 smithi115 ceph-mon[29603]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T18:18:01.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:01 smithi115 ceph-mon[29603]: monmap e2: 2 mons at {a=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0],b=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:18:01.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:01 smithi115 ceph-mon[29603]: fsmap 2024-09-10T18:18:01.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:01 smithi115 ceph-mon[29603]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:18:01.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:01 smithi115 ceph-mon[29603]: mgrmap e13: y(active, since 39s) 2024-09-10T18:18:01.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:01 smithi115 ceph-mon[29603]: overall HEALTH_OK 2024-09-10T18:18:01.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:01 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:01.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:01 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:01.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:01 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:01.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:01 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:18:01.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:01 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:01.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:01 smithi150 ceph-mon[32761]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:01.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:01 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:18:01.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:01 smithi150 ceph-mon[32761]: mon.a calling monitor election 2024-09-10T18:18:01.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:01 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:18:01.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:01 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:18:01.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:01 smithi150 ceph-mon[32761]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:01.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:01 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:18:01.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:01 smithi150 ceph-mon[32761]: mon.b calling monitor election 2024-09-10T18:18:01.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:01 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:18:01.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:01 smithi150 ceph-mon[32761]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:01.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:01 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:18:01.807 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:01 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:18:01.807 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:01 smithi150 ceph-mon[32761]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T18:18:01.807 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:01 smithi150 ceph-mon[32761]: monmap e2: 2 mons at {a=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0],b=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:18:01.807 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:01 smithi150 ceph-mon[32761]: fsmap 2024-09-10T18:18:01.807 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:01 smithi150 ceph-mon[32761]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:18:01.807 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:01 smithi150 ceph-mon[32761]: mgrmap e13: y(active, since 39s) 2024-09-10T18:18:01.807 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:01 smithi150 ceph-mon[32761]: overall HEALTH_OK 2024-09-10T18:18:01.807 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:01 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:01.807 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:01 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:01.807 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:01 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:01.807 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:01 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:18:01.808 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:01 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:02.728 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:02 smithi115 ceph-mon[29603]: Deploying daemon mon.c on smithi115 2024-09-10T18:18:02.728 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:02 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:18:02.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:02 smithi150 ceph-mon[32761]: Deploying daemon mon.c on smithi115 2024-09-10T18:18:02.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:02 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:18:02.963 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:18:02.964 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":2,"fsid":"a600e180-6fa0-11ef-bcea-c7b262605968","modified":"2024-09-10T18:17:56.340898Z","created":"2024-09-10T18:16:23.954068Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3300","nonce":0},{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:3300","nonce":0},{"type":"v1","addr":"172.21.15.150:6789","nonce":0}]},"addr":"172.21.15.150:6789/0","public_addr":"172.21.15.150:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T18:18:02.964 INFO:teuthology.orchestra.run.smithi150.stderr:dumped monmap epoch 2 2024-09-10T18:18:03.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:03 smithi150 ceph-mon[32761]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:03.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:03 smithi150 ceph-mon[32761]: from='client.? 172.21.15.150:0/2205114777' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:18:03.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:03 smithi115 ceph-mon[29603]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:03.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:03 smithi115 ceph-mon[29603]: from='client.? 172.21.15.150:0/2205114777' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:18:04.484 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 podman[41035]: 2024-09-10 18:18:04.343450848 +0000 UTC m=+0.144733127 container create fa822934dbab3984321369afe7084bb1560637c7e78ecf8053d45112bf59b548 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-10T18:18:04.618 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T18:18:04.618 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T18:18:04.866 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:18:04.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 podman[41035]: 2024-09-10 18:18:04.570712377 +0000 UTC m=+0.371994656 container init fa822934dbab3984321369afe7084bb1560637c7e78ecf8053d45112bf59b548 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:18:04.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 podman[41035]: 2024-09-10 18:18:04.575382066 +0000 UTC m=+0.376664346 container start fa822934dbab3984321369afe7084bb1560637c7e78ecf8053d45112bf59b548 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-c, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-10T18:18:04.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T18:18:04.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable), process ceph-mon, pid 7 2024-09-10T18:18:04.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: pidfile_write: ignore empty --pid-file 2024-09-10T18:18:04.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: load: jerasure load: lrc 2024-09-10T18:18:04.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: RocksDB version: 6.15.5 2024-09-10T18:18:04.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-10T18:18:04.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Compile date Sep 9 2024 2024-09-10T18:18:04.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: DB SUMMARY 2024-09-10T18:18:04.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: DB Session ID: OL7HBHG2BYFWK7RKO6KZ 2024-09-10T18:18:04.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: CURRENT file: CURRENT 2024-09-10T18:18:04.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T18:18:04.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-09-10T18:18:04.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-10T18:18:04.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 636 ; 2024-09-10T18:18:04.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.error_if_exists: 0 2024-09-10T18:18:04.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.create_if_missing: 0 2024-09-10T18:18:04.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.paranoid_checks: 1 2024-09-10T18:18:04.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T18:18:04.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.env: 0x5599a8bf4440 2024-09-10T18:18:04.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.fs: Posix File System 2024-09-10T18:18:04.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.info_log: 0x5599a94daec0 2024-09-10T18:18:04.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T18:18:04.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.statistics: (nil) 2024-09-10T18:18:04.989 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.use_fsync: 0 2024-09-10T18:18:04.989 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.max_log_file_size: 0 2024-09-10T18:18:04.989 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T18:18:04.989 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T18:18:04.989 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T18:18:04.989 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T18:18:04.989 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.allow_fallocate: 1 2024-09-10T18:18:04.989 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T18:18:04.989 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T18:18:04.989 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.use_direct_reads: 0 2024-09-10T18:18:04.989 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T18:18:04.989 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T18:18:04.989 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.db_log_dir: 2024-09-10T18:18:04.990 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-c/store.db 2024-09-10T18:18:04.990 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T18:18:04.990 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T18:18:04.990 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T18:18:04.990 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T18:18:04.990 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T18:18:04.990 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T18:18:04.990 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T18:18:04.990 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T18:18:04.990 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.write_buffer_manager: 0x5599a95987b0 2024-09-10T18:18:04.990 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T18:18:04.990 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-10T18:18:04.991 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T18:18:04.991 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T18:18:04.991 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T18:18:04.991 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T18:18:04.991 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T18:18:04.991 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T18:18:04.991 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T18:18:04.991 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.unordered_write: 0 2024-09-10T18:18:04.991 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T18:18:04.991 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T18:18:04.991 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T18:18:04.991 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T18:18:04.991 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.row_cache: None 2024-09-10T18:18:04.992 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.wal_filter: None 2024-09-10T18:18:04.992 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T18:18:04.992 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T18:18:04.992 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.preserve_deletes: 0 2024-09-10T18:18:04.992 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.two_write_queues: 0 2024-09-10T18:18:04.992 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T18:18:04.992 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.atomic_flush: 0 2024-09-10T18:18:04.992 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T18:18:04.992 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T18:18:04.992 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T18:18:04.992 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.log_readahead_size: 0 2024-09-10T18:18:04.992 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T18:18:04.992 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T18:18:04.993 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T18:18:04.993 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T18:18:04.993 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T18:18:04.993 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T18:18:04.993 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.max_background_jobs: 2 2024-09-10T18:18:04.993 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.max_background_compactions: -1 2024-09-10T18:18:04.993 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.max_subcompactions: 1 2024-09-10T18:18:04.993 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T18:18:04.993 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T18:18:04.993 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T18:18:04.993 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T18:18:04.993 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T18:18:04.993 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T18:18:04.994 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T18:18:04.994 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T18:18:04.994 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.max_open_files: -1 2024-09-10T18:18:04.994 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T18:18:04.994 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T18:18:04.994 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T18:18:04.994 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T18:18:04.994 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.max_background_flushes: -1 2024-09-10T18:18:04.994 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Compression algorithms supported: 2024-09-10T18:18:04.994 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: kZSTD supported: 0 2024-09-10T18:18:04.994 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: kXpressCompression supported: 0 2024-09-10T18:18:04.994 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: kBZip2Compression supported: 0 2024-09-10T18:18:04.994 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T18:18:04.995 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: kLZ4Compression supported: 1 2024-09-10T18:18:04.995 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: kZlibCompression supported: 1 2024-09-10T18:18:04.995 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T18:18:04.995 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: kSnappyCompression supported: 1 2024-09-10T18:18:04.995 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T18:18:04.995 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 2024-09-10T18:18:04.995 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-10T18:18:04.995 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T18:18:04.995 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.merge_operator: 2024-09-10T18:18:04.995 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.compaction_filter: None 2024-09-10T18:18:04.995 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T18:18:04.995 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T18:18:04.995 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T18:18:04.996 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T18:18:04.996 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5599a9481a38) 2024-09-10T18:18:04.996 INFO:journalctl@ceph.mon.c.smithi115.stdout: cache_index_and_filter_blocks: 1 2024-09-10T18:18:04.996 INFO:journalctl@ceph.mon.c.smithi115.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T18:18:04.996 INFO:journalctl@ceph.mon.c.smithi115.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T18:18:04.996 INFO:journalctl@ceph.mon.c.smithi115.stdout: pin_top_level_index_and_filter: 1 2024-09-10T18:18:04.996 INFO:journalctl@ceph.mon.c.smithi115.stdout: index_type: 0 2024-09-10T18:18:04.996 INFO:journalctl@ceph.mon.c.smithi115.stdout: data_block_index_type: 0 2024-09-10T18:18:04.996 INFO:journalctl@ceph.mon.c.smithi115.stdout: index_shortening: 1 2024-09-10T18:18:04.996 INFO:journalctl@ceph.mon.c.smithi115.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T18:18:04.996 INFO:journalctl@ceph.mon.c.smithi115.stdout: hash_index_allow_collision: 1 2024-09-10T18:18:04.996 INFO:journalctl@ceph.mon.c.smithi115.stdout: checksum: 1 2024-09-10T18:18:04.996 INFO:journalctl@ceph.mon.c.smithi115.stdout: no_block_cache: 0 2024-09-10T18:18:04.996 INFO:journalctl@ceph.mon.c.smithi115.stdout: block_cache: 0x5599a94b71f0 2024-09-10T18:18:04.997 INFO:journalctl@ceph.mon.c.smithi115.stdout: block_cache_name: BinnedLRUCache 2024-09-10T18:18:04.997 INFO:journalctl@ceph.mon.c.smithi115.stdout: block_cache_options: 2024-09-10T18:18:04.997 INFO:journalctl@ceph.mon.c.smithi115.stdout: capacity : 536870912 2024-09-10T18:18:04.997 INFO:journalctl@ceph.mon.c.smithi115.stdout: num_shard_bits : 4 2024-09-10T18:18:04.997 INFO:journalctl@ceph.mon.c.smithi115.stdout: strict_capacity_limit : 0 2024-09-10T18:18:04.997 INFO:journalctl@ceph.mon.c.smithi115.stdout: high_pri_pool_ratio: 0.000 2024-09-10T18:18:04.997 INFO:journalctl@ceph.mon.c.smithi115.stdout: block_cache_compressed: (nil) 2024-09-10T18:18:04.997 INFO:journalctl@ceph.mon.c.smithi115.stdout: persistent_cache: (nil) 2024-09-10T18:18:04.997 INFO:journalctl@ceph.mon.c.smithi115.stdout: block_size: 4096 2024-09-10T18:18:04.997 INFO:journalctl@ceph.mon.c.smithi115.stdout: block_size_deviation: 10 2024-09-10T18:18:04.997 INFO:journalctl@ceph.mon.c.smithi115.stdout: block_restart_interval: 16 2024-09-10T18:18:04.997 INFO:journalctl@ceph.mon.c.smithi115.stdout: index_block_restart_interval: 1 2024-09-10T18:18:04.997 INFO:journalctl@ceph.mon.c.smithi115.stdout: metadata_block_size: 4096 2024-09-10T18:18:04.998 INFO:journalctl@ceph.mon.c.smithi115.stdout: partition_filters: 0 2024-09-10T18:18:04.998 INFO:journalctl@ceph.mon.c.smithi115.stdout: use_delta_encoding: 1 2024-09-10T18:18:04.998 INFO:journalctl@ceph.mon.c.smithi115.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-10T18:18:04.998 INFO:journalctl@ceph.mon.c.smithi115.stdout: whole_key_filtering: 1 2024-09-10T18:18:04.998 INFO:journalctl@ceph.mon.c.smithi115.stdout: verify_compression: 0 2024-09-10T18:18:04.998 INFO:journalctl@ceph.mon.c.smithi115.stdout: read_amp_bytes_per_bit: 0 2024-09-10T18:18:04.998 INFO:journalctl@ceph.mon.c.smithi115.stdout: format_version: 4 2024-09-10T18:18:04.998 INFO:journalctl@ceph.mon.c.smithi115.stdout: enable_index_compression: 1 2024-09-10T18:18:04.998 INFO:journalctl@ceph.mon.c.smithi115.stdout: block_align: 0 2024-09-10T18:18:04.998 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T18:18:04.998 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T18:18:04.998 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.compression: NoCompression 2024-09-10T18:18:04.998 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T18:18:04.998 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T18:18:04.999 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T18:18:04.999 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.num_levels: 7 2024-09-10T18:18:04.999 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T18:18:04.999 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T18:18:04.999 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T18:18:04.999 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T18:18:04.999 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T18:18:04.999 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T18:18:04.999 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T18:18:04.999 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:18:04.999 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T18:18:04.999 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T18:18:04.999 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T18:18:04.999 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T18:18:05.000 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T18:18:05.000 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T18:18:05.000 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:18:05.000 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T18:18:05.000 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T18:18:05.000 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T18:18:05.000 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T18:18:05.000 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T18:18:05.000 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T18:18:05.000 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T18:18:05.000 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T18:18:05.000 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T18:18:05.000 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T18:18:05.001 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T18:18:05.001 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T18:18:05.001 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T18:18:05.001 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T18:18:05.001 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T18:18:05.001 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T18:18:05.001 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T18:18:05.001 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T18:18:05.001 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T18:18:05.001 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.arena_block_size: 4194304 2024-09-10T18:18:05.001 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T18:18:05.001 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T18:18:05.001 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-10T18:18:05.001 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T18:18:05.002 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T18:18:05.002 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T18:18:05.002 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T18:18:05.002 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T18:18:05.002 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T18:18:05.002 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T18:18:05.002 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T18:18:05.002 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T18:18:05.002 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T18:18:05.002 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T18:18:05.002 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T18:18:05.002 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.inplace_update_support: 0 2024-09-10T18:18:05.002 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T18:18:05.003 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T18:18:05.003 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T18:18:05.003 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T18:18:05.003 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.bloom_locality: 0 2024-09-10T18:18:05.003 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.max_successive_merges: 0 2024-09-10T18:18:05.003 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T18:18:05.003 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T18:18:05.003 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T18:18:05.003 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T18:18:05.003 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.ttl: 2592000 2024-09-10T18:18:05.003 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T18:18:05.003 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.enable_blob_files: false 2024-09-10T18:18:05.003 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.min_blob_size: 0 2024-09-10T18:18:05.004 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T18:18:05.004 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T18:18:05.004 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T18:18:05.004 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T18:18:05.004 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T18:18:05.004 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-09-10T18:18:05.004 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-09-10T18:18:05.004 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725992284598759, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-10T18:18:05.004 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-09-10T18:18:05.004 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-10T18:18:05.004 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725992284599053, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1696, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1725992284, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "df236a65-21a6-413a-ba7e-6882a0439b9b", "db_session_id": "OL7HBHG2BYFWK7RKO6KZ"}} 2024-09-10T18:18:05.004 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-09-10T18:18:05.004 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725992284599361, "job": 1, "event": "recovery_finished"} 2024-09-10T18:18:05.005 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T18:18:05.005 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x5599a94ea700 2024-09-10T18:18:05.005 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: DB pointer 0x5599a94ba000 2024-09-10T18:18:05.005 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-10T18:18:05.005 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-10T18:18:05.005 INFO:journalctl@ceph.mon.c.smithi115.stdout: ** DB Stats ** 2024-09-10T18:18:05.005 INFO:journalctl@ceph.mon.c.smithi115.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:18:05.005 INFO:journalctl@ceph.mon.c.smithi115.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T18:18:05.005 INFO:journalctl@ceph.mon.c.smithi115.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T18:18:05.005 INFO:journalctl@ceph.mon.c.smithi115.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:18:05.005 INFO:journalctl@ceph.mon.c.smithi115.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T18:18:05.005 INFO:journalctl@ceph.mon.c.smithi115.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-10T18:18:05.005 INFO:journalctl@ceph.mon.c.smithi115.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:18:05.005 INFO:journalctl@ceph.mon.c.smithi115.stdout: 2024-09-10T18:18:05.006 INFO:journalctl@ceph.mon.c.smithi115.stdout: ** Compaction Stats [default] ** 2024-09-10T18:18:05.006 INFO:journalctl@ceph.mon.c.smithi115.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:18:05.006 INFO:journalctl@ceph.mon.c.smithi115.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:18:05.006 INFO:journalctl@ceph.mon.c.smithi115.stdout: L0 1/0 1.66 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.8 0.00 0.00 1 0.000 0 0 2024-09-10T18:18:05.006 INFO:journalctl@ceph.mon.c.smithi115.stdout: Sum 1/0 1.66 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.8 0.00 0.00 1 0.000 0 0 2024-09-10T18:18:05.006 INFO:journalctl@ceph.mon.c.smithi115.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.8 0.00 0.00 1 0.000 0 0 2024-09-10T18:18:05.006 INFO:journalctl@ceph.mon.c.smithi115.stdout: 2024-09-10T18:18:05.006 INFO:journalctl@ceph.mon.c.smithi115.stdout: ** Compaction Stats [default] ** 2024-09-10T18:18:05.006 INFO:journalctl@ceph.mon.c.smithi115.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:18:05.006 INFO:journalctl@ceph.mon.c.smithi115.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:18:05.006 INFO:journalctl@ceph.mon.c.smithi115.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.8 0.00 0.00 1 0.000 0 0 2024-09-10T18:18:05.006 INFO:journalctl@ceph.mon.c.smithi115.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:18:05.006 INFO:journalctl@ceph.mon.c.smithi115.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:18:05.007 INFO:journalctl@ceph.mon.c.smithi115.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:18:05.007 INFO:journalctl@ceph.mon.c.smithi115.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:18:05.007 INFO:journalctl@ceph.mon.c.smithi115.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:18:05.007 INFO:journalctl@ceph.mon.c.smithi115.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:18:05.007 INFO:journalctl@ceph.mon.c.smithi115.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:18:05.007 INFO:journalctl@ceph.mon.c.smithi115.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:18:05.007 INFO:journalctl@ceph.mon.c.smithi115.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:18:05.007 INFO:journalctl@ceph.mon.c.smithi115.stdout: 2024-09-10T18:18:05.007 INFO:journalctl@ceph.mon.c.smithi115.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:18:05.007 INFO:journalctl@ceph.mon.c.smithi115.stdout: 2024-09-10T18:18:05.007 INFO:journalctl@ceph.mon.c.smithi115.stdout: ** Compaction Stats [default] ** 2024-09-10T18:18:05.007 INFO:journalctl@ceph.mon.c.smithi115.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:18:05.007 INFO:journalctl@ceph.mon.c.smithi115.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:18:05.008 INFO:journalctl@ceph.mon.c.smithi115.stdout: L0 1/0 1.66 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.8 0.00 0.00 1 0.000 0 0 2024-09-10T18:18:05.008 INFO:journalctl@ceph.mon.c.smithi115.stdout: Sum 1/0 1.66 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.8 0.00 0.00 1 0.000 0 0 2024-09-10T18:18:05.008 INFO:journalctl@ceph.mon.c.smithi115.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-10T18:18:05.008 INFO:journalctl@ceph.mon.c.smithi115.stdout: 2024-09-10T18:18:05.008 INFO:journalctl@ceph.mon.c.smithi115.stdout: ** Compaction Stats [default] ** 2024-09-10T18:18:05.008 INFO:journalctl@ceph.mon.c.smithi115.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:18:05.008 INFO:journalctl@ceph.mon.c.smithi115.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:18:05.008 INFO:journalctl@ceph.mon.c.smithi115.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.8 0.00 0.00 1 0.000 0 0 2024-09-10T18:18:05.008 INFO:journalctl@ceph.mon.c.smithi115.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:18:05.008 INFO:journalctl@ceph.mon.c.smithi115.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:18:05.009 INFO:journalctl@ceph.mon.c.smithi115.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:18:05.009 INFO:journalctl@ceph.mon.c.smithi115.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:18:05.009 INFO:journalctl@ceph.mon.c.smithi115.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:18:05.009 INFO:journalctl@ceph.mon.c.smithi115.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:18:05.009 INFO:journalctl@ceph.mon.c.smithi115.stdout: Cumulative compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:18:05.009 INFO:journalctl@ceph.mon.c.smithi115.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:18:05.009 INFO:journalctl@ceph.mon.c.smithi115.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:18:05.009 INFO:journalctl@ceph.mon.c.smithi115.stdout: 2024-09-10T18:18:05.009 INFO:journalctl@ceph.mon.c.smithi115.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:18:05.009 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-10T18:18:05.009 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: using public_addrv [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] 2024-09-10T18:18:05.009 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: starting mon.c rank -1 at public addrs [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] at bind addrs [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid a600e180-6fa0-11ef-bcea-c7b262605968 2024-09-10T18:18:05.009 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: mon.c@-1(???) e0 preinit fsid a600e180-6fa0-11ef-bcea-c7b262605968 2024-09-10T18:18:05.010 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: mon.c@-1(synchronizing).mds e1 new map 2024-09-10T18:18:05.010 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-10T18:18:05.010 INFO:journalctl@ceph.mon.c.smithi115.stdout: e1 2024-09-10T18:18:05.010 INFO:journalctl@ceph.mon.c.smithi115.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T18:18:05.010 INFO:journalctl@ceph.mon.c.smithi115.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T18:18:05.010 INFO:journalctl@ceph.mon.c.smithi115.stdout: legacy client fscid: -1 2024-09-10T18:18:05.010 INFO:journalctl@ceph.mon.c.smithi115.stdout: 2024-09-10T18:18:05.010 INFO:journalctl@ceph.mon.c.smithi115.stdout: No filesystems configured 2024-09-10T18:18:05.010 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-10T18:18:05.011 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-10T18:18:05.011 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-10T18:18:05.011 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-10T18:18:05.011 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-10T18:18:05.011 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-10T18:18:05.012 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T18:18:05.012 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:18:05.012 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:18:05.012 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:18:05.012 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: Updating smithi150:/etc/ceph/ceph.conf 2024-09-10T18:18:05.012 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: Updating smithi150:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:18:05.012 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:18:05.012 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:05.012 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:05.012 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:05.013 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: Updating smithi150:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:18:05.013 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:05.013 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/3670855908' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T18:18:05.013 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/3670855908' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T18:18:05.013 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:18:05.013 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:05.013 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi115:172.21.15.115=a;smithi115:[v2:172.21.15.115:3301,v1:172.21.15.115:6790]=c;smithi150:172.21.15.150=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:18:05.013 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: Saving service mon spec with placement smithi115:172.21.15.115=a;smithi115:[v2:172.21.15.115:3301,v1:172.21.15.115:6790]=c;smithi150:172.21.15.150=b;count:3 2024-09-10T18:18:05.013 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:05.013 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:18:05.013 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:05.013 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:18:05.013 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:05.014 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:18:05.014 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:05.014 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: Deploying daemon mon.b on smithi150 2024-09-10T18:18:05.014 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:05.014 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:05.014 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:05.014 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:18:05.014 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: mon.a calling monitor election 2024-09-10T18:18:05.015 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:18:05.015 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:18:05.015 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:05.015 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:18:05.015 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: mon.b calling monitor election 2024-09-10T18:18:05.015 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:18:05.015 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:05.015 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:18:05.015 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:18:05.016 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T18:18:05.016 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: monmap e2: 2 mons at {a=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0],b=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:18:05.016 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: fsmap 2024-09-10T18:18:05.016 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:18:05.016 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: mgrmap e13: y(active, since 39s) 2024-09-10T18:18:05.016 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: overall HEALTH_OK 2024-09-10T18:18:05.016 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:05.016 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:05.016 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:05.017 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:18:05.017 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:05.017 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: Deploying daemon mon.c on smithi115 2024-09-10T18:18:05.017 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:18:05.017 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:05.017 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: from='client.? 172.21.15.150:0/2205114777' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:18:05.017 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 ceph-mon[41073]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-10T18:18:05.017 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 bash[41035]: fa822934dbab3984321369afe7084bb1560637c7e78ecf8053d45112bf59b548 2024-09-10T18:18:05.017 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:04 smithi115 systemd[1]: Started Ceph mon.c for a600e180-6fa0-11ef-bcea-c7b262605968. 2024-09-10T18:18:09.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:18:09.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[41073]: mon.a calling monitor election 2024-09-10T18:18:09.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:18:09.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:18:09.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[41073]: mon.b calling monitor election 2024-09-10T18:18:09.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:18:09.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[41073]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:09.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:18:09.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[41073]: mon.c calling monitor election 2024-09-10T18:18:09.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:18:09.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[41073]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:09.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:18:09.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:18:09.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[41073]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-10T18:18:09.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[41073]: monmap e3: 3 mons at {a=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0],b=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0],c=[v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:18:09.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[41073]: fsmap 2024-09-10T18:18:09.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[41073]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:18:09.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[41073]: mgrmap e13: y(active, since 47s) 2024-09-10T18:18:09.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[41073]: overall HEALTH_OK 2024-09-10T18:18:09.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:09.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:09.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:09.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:09.987 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:18:09.987 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[29603]: mon.a calling monitor election 2024-09-10T18:18:09.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:18:09.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:18:09.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[29603]: mon.b calling monitor election 2024-09-10T18:18:09.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:18:09.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[29603]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:09.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:18:09.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[29603]: mon.c calling monitor election 2024-09-10T18:18:09.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:18:09.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[29603]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:09.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:18:09.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:18:09.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[29603]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-10T18:18:09.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[29603]: monmap e3: 3 mons at {a=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0],b=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0],c=[v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:18:09.989 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[29603]: fsmap 2024-09-10T18:18:09.989 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[29603]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:18:09.989 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[29603]: mgrmap e13: y(active, since 47s) 2024-09-10T18:18:09.989 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[29603]: overall HEALTH_OK 2024-09-10T18:18:09.989 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:09.989 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:09.989 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:09.989 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:09 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:10.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:09 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:18:10.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:09 smithi150 ceph-mon[32761]: mon.a calling monitor election 2024-09-10T18:18:10.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:09 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:18:10.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:09 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:18:10.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:09 smithi150 ceph-mon[32761]: mon.b calling monitor election 2024-09-10T18:18:10.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:09 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:18:10.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:09 smithi150 ceph-mon[32761]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:10.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:09 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:18:10.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:09 smithi150 ceph-mon[32761]: mon.c calling monitor election 2024-09-10T18:18:10.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:09 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:18:10.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:09 smithi150 ceph-mon[32761]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:10.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:09 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:18:10.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:09 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:18:10.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:09 smithi150 ceph-mon[32761]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-10T18:18:10.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:09 smithi150 ceph-mon[32761]: monmap e3: 3 mons at {a=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0],b=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0],c=[v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:18:10.057 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:09 smithi150 ceph-mon[32761]: fsmap 2024-09-10T18:18:10.057 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:09 smithi150 ceph-mon[32761]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:18:10.057 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:09 smithi150 ceph-mon[32761]: mgrmap e13: y(active, since 47s) 2024-09-10T18:18:10.057 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:09 smithi150 ceph-mon[32761]: overall HEALTH_OK 2024-09-10T18:18:10.057 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:09 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:10.057 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:09 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:10.057 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:09 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:10.057 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:09 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:10.179 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T18:18:10.179 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":3,"fsid":"a600e180-6fa0-11ef-bcea-c7b262605968","modified":"2024-09-10T18:18:04.622443Z","created":"2024-09-10T18:16:23.954068Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3300","nonce":0},{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:3300","nonce":0},{"type":"v1","addr":"172.21.15.150:6789","nonce":0}]},"addr":"172.21.15.150:6789/0","public_addr":"172.21.15.150:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3301","nonce":0},{"type":"v1","addr":"172.21.15.115:6790","nonce":0}]},"addr":"172.21.15.115:6790/0","public_addr":"172.21.15.115:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-10T18:18:10.180 INFO:teuthology.orchestra.run.smithi150.stderr:dumped monmap epoch 3 2024-09-10T18:18:10.829 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-10T18:18:10.829 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-10T18:18:10.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:10 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:18:10.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:10 smithi150 ceph-mon[32761]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:10.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:10 smithi150 ceph-mon[32761]: from='client.? 172.21.15.150:0/3300053040' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:18:10.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:10 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:18:10.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:10 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:18:10.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:10 smithi115 ceph-mon[29603]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:10.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:10 smithi115 ceph-mon[29603]: from='client.? 172.21.15.150:0/3300053040' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:18:10.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:10 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:18:10.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:10 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:18:10.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:10 smithi115 ceph-mon[41073]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:10.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:10 smithi115 ceph-mon[41073]: from='client.? 172.21.15.150:0/3300053040' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:18:10.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:10 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:18:11.067 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:18:12.200 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:11 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:12.200 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:11 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:12.200 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:11 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:12.200 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:11 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:12.200 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:11 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:12.200 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:11 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:18:12.201 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:11 smithi115 ceph-mon[29603]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-10T18:18:12.201 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:11 smithi115 ceph-mon[29603]: Updating smithi150:/etc/ceph/ceph.conf 2024-09-10T18:18:12.201 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:11 smithi115 ceph-mon[29603]: Updating smithi115:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:18:12.201 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:11 smithi115 ceph-mon[29603]: Updating smithi150:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:18:12.201 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:11 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:12.201 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:11 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:12.201 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:11 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:12.201 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:11 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:12.201 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:11 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:12.201 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:11 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:18:12.201 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:11 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:18:12.201 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:11 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:12.202 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:11 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:12.202 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:11 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:12.202 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:11 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:12.202 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:11 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:12.202 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:11 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:12.202 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:11 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:18:12.202 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:11 smithi115 ceph-mon[41073]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-10T18:18:12.202 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:11 smithi115 ceph-mon[41073]: Updating smithi150:/etc/ceph/ceph.conf 2024-09-10T18:18:12.202 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:11 smithi115 ceph-mon[41073]: Updating smithi115:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:18:12.203 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:11 smithi115 ceph-mon[41073]: Updating smithi150:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:18:12.203 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:11 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:12.203 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:11 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:12.203 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:11 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:12.203 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:11 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:12.203 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:11 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:12.203 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:11 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:18:12.203 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:11 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:18:12.203 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:11 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:12.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:11 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:12.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:11 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:12.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:11 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:12.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:11 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:12.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:11 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:12.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:11 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:18:12.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:11 smithi150 ceph-mon[32761]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-10T18:18:12.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:11 smithi150 ceph-mon[32761]: Updating smithi150:/etc/ceph/ceph.conf 2024-09-10T18:18:12.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:11 smithi150 ceph-mon[32761]: Updating smithi115:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:18:12.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:11 smithi150 ceph-mon[32761]: Updating smithi150:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:18:12.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:11 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:12.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:11 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:12.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:11 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:12.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:11 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:12.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:11 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:12.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:11 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:18:12.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:11 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:18:12.307 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:11 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:12.871 INFO:teuthology.orchestra.run.smithi115.stdout:# minimal ceph.conf for a600e180-6fa0-11ef-bcea-c7b262605968 2024-09-10T18:18:12.871 INFO:teuthology.orchestra.run.smithi115.stdout:[global] 2024-09-10T18:18:12.871 INFO:teuthology.orchestra.run.smithi115.stdout: fsid = a600e180-6fa0-11ef-bcea-c7b262605968 2024-09-10T18:18:12.871 INFO:teuthology.orchestra.run.smithi115.stdout: mon_host = [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] 2024-09-10T18:18:13.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:13 smithi150 ceph-mon[32761]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T18:18:13.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:13 smithi150 ceph-mon[32761]: Reconfiguring daemon mon.a on smithi115 2024-09-10T18:18:13.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:13 smithi150 ceph-mon[32761]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:13.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:13 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:13.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:13 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/3941126689' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:13.325 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:13 smithi115 ceph-mon[41073]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T18:18:13.325 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:13 smithi115 ceph-mon[41073]: Reconfiguring daemon mon.a on smithi115 2024-09-10T18:18:13.326 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:13 smithi115 ceph-mon[41073]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:13.326 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:13 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:13.326 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:13 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/3941126689' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:13.326 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:13 smithi115 ceph-mon[29603]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T18:18:13.326 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:13 smithi115 ceph-mon[29603]: Reconfiguring daemon mon.a on smithi115 2024-09-10T18:18:13.326 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:13 smithi115 ceph-mon[29603]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:13.326 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:13 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:13.326 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:13 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/3941126689' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:13.931 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-10T18:18:13.931 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T18:18:13.931 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T18:18:13.962 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T18:18:13.962 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:18:14.030 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-10T18:18:14.030 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T18:18:14.063 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-10T18:18:14.064 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:18:14.132 INFO:tasks.cephadm:Adding mgr.y on smithi115 2024-09-10T18:18:14.132 INFO:tasks.cephadm:Adding mgr.x on smithi150 2024-09-10T18:18:14.132 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph orch apply mgr '2;smithi115=y;smithi150=x' 2024-09-10T18:18:14.411 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:18:15.617 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:15 smithi150 ceph-mon[32761]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:15.617 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:15 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:15.617 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:15 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:15.617 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:15 smithi150 ceph-mon[32761]: Reconfiguring mon.c (monmap changed)... 2024-09-10T18:18:15.617 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:15 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:18:15.617 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:15 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:18:15.617 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:15 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:15.618 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:15 smithi150 ceph-mon[32761]: Reconfiguring daemon mon.c on smithi115 2024-09-10T18:18:15.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:15 smithi115 ceph-mon[29603]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:15.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:15 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:15.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:15 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:15.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:15 smithi115 ceph-mon[29603]: Reconfiguring mon.c (monmap changed)... 2024-09-10T18:18:15.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:15 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:18:15.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:15 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:18:15.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:15 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:15.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:15 smithi115 ceph-mon[29603]: Reconfiguring daemon mon.c on smithi115 2024-09-10T18:18:15.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:15 smithi115 ceph-mon[41073]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:15.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:15 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:15.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:15 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:15.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:15 smithi115 ceph-mon[41073]: Reconfiguring mon.c (monmap changed)... 2024-09-10T18:18:15.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:15 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:18:15.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:15 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:18:15.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:15 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:15.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:15 smithi115 ceph-mon[41073]: Reconfiguring daemon mon.c on smithi115 2024-09-10T18:18:15.980 INFO:teuthology.orchestra.run.smithi150.stdout:Scheduled mgr update... 2024-09-10T18:18:16.570 DEBUG:teuthology.orchestra.run.smithi150:mgr.x> sudo journalctl -f -n 0 -u ceph-a600e180-6fa0-11ef-bcea-c7b262605968@mgr.x.service 2024-09-10T18:18:16.573 INFO:tasks.cephadm:Deploying OSDs... 2024-09-10T18:18:16.573 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T18:18:16.573 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T18:18:16.591 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T18:18:16.592 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_1 2024-09-10T18:18:16.648 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T18:18:16.648 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:18:16.649 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 974 Links: 1 2024-09-10T18:18:16.649 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:18:16.649 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:18:16.649 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-10 18:18:04.042200448 +0000 2024-09-10T18:18:16.649 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-10 18:13:13.858454582 +0000 2024-09-10T18:18:16.649 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-10 18:13:13.858454582 +0000 2024-09-10T18:18:16.649 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-09-10 18:13:13.858454582 +0000 2024-09-10T18:18:16.649 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T18:18:16.714 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-10T18:18:16.714 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-10T18:18:16.714 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000190189 s, 2.7 MB/s 2024-09-10T18:18:16.716 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T18:18:16.772 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_2 2024-09-10T18:18:16.829 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T18:18:16.829 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:18:16.829 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 964 Links: 1 2024-09-10T18:18:16.830 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:18:16.830 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:18:16.830 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-10 18:18:04.043200430 +0000 2024-09-10T18:18:16.830 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-10 18:13:13.858454582 +0000 2024-09-10T18:18:16.830 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-10 18:13:13.858454582 +0000 2024-09-10T18:18:16.830 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-09-10 18:13:13.858454582 +0000 2024-09-10T18:18:16.830 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T18:18:16.894 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-10T18:18:16.894 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-10T18:18:16.894 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000243432 s, 2.1 MB/s 2024-09-10T18:18:16.895 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T18:18:16.952 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_3 2024-09-10T18:18:16.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:16 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:16.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:16 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:16.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:16 smithi115 ceph-mon[41073]: Reconfiguring mon.b (monmap changed)... 2024-09-10T18:18:16.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:16 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:18:16.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:16 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:18:16.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:16 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:16.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:16 smithi115 ceph-mon[41073]: Reconfiguring daemon mon.b on smithi150 2024-09-10T18:18:16.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:16 smithi115 ceph-mon[41073]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi115=y;smithi150=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:18:16.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:16 smithi115 ceph-mon[41073]: Saving service mgr spec with placement smithi115=y;smithi150=x;count:2 2024-09-10T18:18:16.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:16 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:16.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:16 smithi115 ceph-mon[41073]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:16.986 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:16 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:16.986 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:16 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:16.986 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:16 smithi115 ceph-mon[29603]: Reconfiguring mon.b (monmap changed)... 2024-09-10T18:18:16.986 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:16 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:18:16.986 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:16 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:18:16.986 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:16 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:16.986 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:16 smithi115 ceph-mon[29603]: Reconfiguring daemon mon.b on smithi150 2024-09-10T18:18:16.987 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:16 smithi115 ceph-mon[29603]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi115=y;smithi150=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:18:16.987 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:16 smithi115 ceph-mon[29603]: Saving service mgr spec with placement smithi115=y;smithi150=x;count:2 2024-09-10T18:18:16.987 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:16 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:16.987 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:16 smithi115 ceph-mon[29603]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:17.002 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T18:18:17.002 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:18:17.002 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 984 Links: 1 2024-09-10T18:18:17.002 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:18:17.002 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:18:17.002 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-10 18:18:04.043200430 +0000 2024-09-10T18:18:17.002 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-10 18:13:13.859454564 +0000 2024-09-10T18:18:17.002 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-10 18:13:13.859454564 +0000 2024-09-10T18:18:17.002 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-09-10 18:13:13.859454564 +0000 2024-09-10T18:18:17.003 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T18:18:17.066 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-10T18:18:17.066 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-10T18:18:17.066 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000222339 s, 2.3 MB/s 2024-09-10T18:18:17.067 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T18:18:17.123 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_4 2024-09-10T18:18:17.179 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T18:18:17.179 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:18:17.179 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 973 Links: 1 2024-09-10T18:18:17.179 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:18:17.179 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:18:17.179 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-10 18:18:04.043200430 +0000 2024-09-10T18:18:17.179 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-10 18:13:13.858454582 +0000 2024-09-10T18:18:17.179 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-10 18:13:13.858454582 +0000 2024-09-10T18:18:17.179 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-09-10 18:13:13.858454582 +0000 2024-09-10T18:18:17.180 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T18:18:17.242 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-10T18:18:17.242 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-10T18:18:17.242 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000214561 s, 2.4 MB/s 2024-09-10T18:18:17.243 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T18:18:17.276 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:16 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:17.276 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:16 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:17.276 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:16 smithi150 ceph-mon[32761]: Reconfiguring mon.b (monmap changed)... 2024-09-10T18:18:17.276 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:16 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:18:17.276 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:16 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:18:17.276 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:16 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:17.276 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:16 smithi150 ceph-mon[32761]: Reconfiguring daemon mon.b on smithi150 2024-09-10T18:18:17.277 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:16 smithi150 ceph-mon[32761]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi115=y;smithi150=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:18:17.277 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:16 smithi150 ceph-mon[32761]: Saving service mgr spec with placement smithi115=y;smithi150=x;count:2 2024-09-10T18:18:17.277 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:16 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:17.277 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:16 smithi150 ceph-mon[32761]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:17.300 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-10T18:18:17.300 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T18:18:17.316 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T18:18:17.316 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_1 2024-09-10T18:18:17.372 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T18:18:17.372 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:18:17.372 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 972 Links: 1 2024-09-10T18:18:17.372 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:18:17.372 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:18:17.372 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-09-10 18:17:55.144994932 +0000 2024-09-10T18:18:17.372 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-09-10 18:13:15.523491539 +0000 2024-09-10T18:18:17.372 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-09-10 18:13:15.523491539 +0000 2024-09-10T18:18:17.372 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-09-10 18:13:15.523491539 +0000 2024-09-10T18:18:17.372 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T18:18:17.437 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-09-10T18:18:17.437 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-09-10T18:18:17.437 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.00024715 s, 2.1 MB/s 2024-09-10T18:18:17.438 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T18:18:17.494 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_2 2024-09-10T18:18:17.549 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T18:18:17.549 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:18:17.550 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 978 Links: 1 2024-09-10T18:18:17.550 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:18:17.550 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:18:17.550 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-09-10 18:17:55.144994932 +0000 2024-09-10T18:18:17.550 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-09-10 18:13:15.525491500 +0000 2024-09-10T18:18:17.550 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-09-10 18:13:15.525491500 +0000 2024-09-10T18:18:17.550 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-09-10 18:13:15.525491500 +0000 2024-09-10T18:18:17.550 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T18:18:17.614 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-09-10T18:18:17.614 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-09-10T18:18:17.614 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000820114 s, 624 kB/s 2024-09-10T18:18:17.616 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T18:18:17.672 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_3 2024-09-10T18:18:17.728 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T18:18:17.728 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:18:17.728 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 963 Links: 1 2024-09-10T18:18:17.728 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:18:17.728 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:18:17.728 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-09-10 18:17:55.144994932 +0000 2024-09-10T18:18:17.728 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-09-10 18:13:15.522491559 +0000 2024-09-10T18:18:17.729 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-09-10 18:13:15.522491559 +0000 2024-09-10T18:18:17.729 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-09-10 18:13:15.522491559 +0000 2024-09-10T18:18:17.729 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T18:18:17.792 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-09-10T18:18:17.793 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-09-10T18:18:17.793 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.00021766 s, 2.4 MB/s 2024-09-10T18:18:17.794 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T18:18:17.850 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_4 2024-09-10T18:18:17.906 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T18:18:17.906 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:18:17.906 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 984 Links: 1 2024-09-10T18:18:17.906 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:18:17.906 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:18:17.906 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-09-10 18:17:55.145994912 +0000 2024-09-10T18:18:17.906 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-09-10 18:13:15.525491500 +0000 2024-09-10T18:18:17.906 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-09-10 18:13:15.525491500 +0000 2024-09-10T18:18:17.907 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-09-10 18:13:15.525491500 +0000 2024-09-10T18:18:17.907 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T18:18:17.970 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-09-10T18:18:17.970 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-09-10T18:18:17.971 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.00019416 s, 2.6 MB/s 2024-09-10T18:18:17.972 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T18:18:18.028 INFO:tasks.cephadm:Deploying osd.0 on smithi115 with /dev/vg_nvme/lv_4... 2024-09-10T18:18:18.028 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-10T18:18:18.270 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:18:18.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:18 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:18.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:18 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:18.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:18 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:18:18.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:18 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:18.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:18 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:18:18.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:18 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:18.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:18 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:18:18.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:18 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T18:18:18.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:18 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:18:18.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:18 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:18.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:18 smithi115 ceph-mon[29603]: Deploying daemon mgr.x on smithi150 2024-09-10T18:18:18.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:18 smithi115 ceph-mon[29603]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:18.646 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:18 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:18.646 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:18 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:18.646 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:18 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:18:18.646 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:18 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:18.647 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:18 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:18:18.647 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:18 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:18.647 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:18 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:18:18.647 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:18 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T18:18:18.647 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:18 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:18:18.647 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:18 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:18.647 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:18 smithi115 ceph-mon[41073]: Deploying daemon mgr.x on smithi150 2024-09-10T18:18:18.647 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:18 smithi115 ceph-mon[41073]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:18.996 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:18 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:18.996 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:18 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:18.996 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:18 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:18:18.996 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:18 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:18.996 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:18 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:18:18.996 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:18 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:18.997 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:18 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:18:18.997 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:18 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T18:18:18.997 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:18 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:18:18.997 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:18 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:18.997 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:18 smithi150 ceph-mon[32761]: Deploying daemon mgr.x on smithi150 2024-09-10T18:18:18.997 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:18 smithi150 ceph-mon[32761]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:19.305 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:18:19 smithi150 systemd[1]: Starting Ceph mgr.x for a600e180-6fa0-11ef-bcea-c7b262605968... 2024-09-10T18:18:19.679 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:18:19 smithi150 podman[34800]: 2024-09-10 18:18:19.276675708 +0000 UTC m=+0.015782316 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:18:19.679 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:18:19 smithi150 podman[34800]: 2024-09-10 18:18:19.391931147 +0000 UTC m=+0.131037789 container create 5f889455cd5022c28ace2a5f7eb56131b238ec0523b57a5a8ddb7c3787119c5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-10T18:18:19.679 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:18:19 smithi150 podman[34800]: 2024-09-10 18:18:19.59401721 +0000 UTC m=+0.333123809 container init 5f889455cd5022c28ace2a5f7eb56131b238ec0523b57a5a8ddb7c3787119c5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T18:18:19.679 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:18:19 smithi150 podman[34800]: 2024-09-10 18:18:19.598687539 +0000 UTC m=+0.337794141 container start 5f889455cd5022c28ace2a5f7eb56131b238ec0523b57a5a8ddb7c3787119c5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:18:19.679 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:18:19 smithi150 ceph-mgr[34838]: -- 172.21.15.150:0/2615909782 <== mon.1 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5602f7482340 con 0x5602f748e800 2024-09-10T18:18:19.679 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:18:19 smithi150 bash[34800]: 5f889455cd5022c28ace2a5f7eb56131b238ec0523b57a5a8ddb7c3787119c5b 2024-09-10T18:18:19.979 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-10T18:18:19.979 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:18:19 smithi150 systemd[1]: Started Ceph mgr.x for a600e180-6fa0-11ef-bcea-c7b262605968. 2024-09-10T18:18:19.980 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:18:19 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:18:19.718+0000 7fb552ce4ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:18:19.980 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:18:19 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:18:19.779+0000 7fb552ce4ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:18:19.980 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:18:19 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:18:19.847+0000 7fb552ce4ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:18:19.980 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:18:19 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:18:19.977+0000 7fb552ce4ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:18:20.061 INFO:teuthology.orchestra.run.smithi115.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-10T18:18:20.074 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10+0 records in 2024-09-10T18:18:20.075 INFO:teuthology.orchestra.run.smithi115.stderr:10+0 records out 2024-09-10T18:18:20.075 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0117173 s, 895 MB/s 2024-09-10T18:18:20.075 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping successful for: 2024-09-10T18:18:20.305 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:18:20 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:18:20.125+0000 7fb552ce4ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:18:20.672 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi115:vg_nvme/lv_4 2024-09-10T18:18:20.748 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:18:20 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:18:20.400+0000 7fb552ce4ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:18:20.748 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:18:20 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:18:20.463+0000 7fb552ce4ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:18:20.748 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:18:20 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:18:20.630+0000 7fb552ce4ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:18:20.919 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:18:20.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:20 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:20.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:20 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:20.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:20 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:20.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:20 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:20.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:20 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:18:20.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:20 smithi115 ceph-mon[29603]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:20.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:20 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:20.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:20 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:20.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:20 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:20.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:20 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:20.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:20 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:18:20.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:20 smithi115 ceph-mon[41073]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:21.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:20 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:21.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:20 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:21.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:20 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:21.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:20 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:21.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:20 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:18:21.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:20 smithi150 ceph-mon[32761]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:21.805 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:18:21 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:18:21.488+0000 7fb552ce4ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:18:21.805 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:18:21 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:18:21.551+0000 7fb552ce4ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:18:22.042 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:21 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:22.042 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:21 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:22.042 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:21 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:22.042 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:21 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:18:22.043 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:21 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:22.043 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:21 smithi115 ceph-mon[29603]: Reconfiguring mgr.y (unknown last config time)... 2024-09-10T18:18:22.043 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:21 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:18:22.043 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:21 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:18:22.043 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:21 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:22.043 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:21 smithi115 ceph-mon[29603]: Reconfiguring daemon mgr.y on smithi115 2024-09-10T18:18:22.043 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:21 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:22.043 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:21 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:22.043 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:21 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:22.043 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:21 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:18:22.044 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:21 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:22.044 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:21 smithi115 ceph-mon[41073]: Reconfiguring mgr.y (unknown last config time)... 2024-09-10T18:18:22.044 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:21 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:18:22.044 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:21 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:18:22.044 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:21 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:22.044 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:21 smithi115 ceph-mon[41073]: Reconfiguring daemon mgr.y on smithi115 2024-09-10T18:18:22.261 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:21 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:22.262 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:21 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:22.262 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:21 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:22.262 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:21 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:18:22.262 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:21 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:22.262 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:21 smithi150 ceph-mon[32761]: Reconfiguring mgr.y (unknown last config time)... 2024-09-10T18:18:22.262 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:21 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:18:22.263 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:21 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:18:22.263 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:21 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:22.263 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:21 smithi150 ceph-mon[32761]: Reconfiguring daemon mgr.y on smithi115 2024-09-10T18:18:22.550 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:18:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:18:22.259+0000 7fb552ce4ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:18:22.550 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:18:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:18:22.370+0000 7fb552ce4ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:18:22.550 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:18:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:18:22.490+0000 7fb552ce4ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:18:22.550 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:18:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:18:22.548+0000 7fb552ce4ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:18:22.805 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:18:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:18:22.674+0000 7fb552ce4ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:18:22.805 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:18:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:18:22.740+0000 7fb552ce4ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:18:22.805 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:18:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:18:22.797+0000 7fb552ce4ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:18:23.195 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:23 smithi150 ceph-mon[32761]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:23.195 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:23 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:23.195 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:18:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:18:22.931+0000 7fb552ce4ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:18:23.195 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:18:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:18:22.993+0000 7fb552ce4ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:18:23.195 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:18:23 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:18:23.058+0000 7fb552ce4ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:18:23.195 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:18:23 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:18:23.137+0000 7fb552ce4ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:18:23.195 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:18:23 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:18:23.193+0000 7fb552ce4ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:18:23.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:23 smithi115 ceph-mon[29603]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:23.416 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:23 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:23.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:23 smithi115 ceph-mon[41073]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:23.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:23 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:23.555 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:18:23 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:18:23.250+0000 7fb552ce4ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:18:24.305 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:18:23 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:18:23.997+0000 7fb552ce4ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:18:24.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:24 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:24.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:24 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:24.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:24 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:18:24.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:24 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:24.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:24 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:18:24.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:24 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:24.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:24 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:18:24.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:24 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:18:24.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:24 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:24.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:24 smithi115 ceph-mon[29603]: Standby manager daemon x started 2024-09-10T18:18:24.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:24 smithi115 ceph-mon[29603]: from='mgr.? 172.21.15.150:0/750740070' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:18:24.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:24 smithi115 ceph-mon[29603]: from='mgr.? 172.21.15.150:0/750740070' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:18:24.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:24 smithi115 ceph-mon[29603]: from='mgr.? 172.21.15.150:0/750740070' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:18:24.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:24 smithi115 ceph-mon[29603]: from='mgr.? 172.21.15.150:0/750740070' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:18:24.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:24 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:24.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:24 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:24.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:24 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:18:24.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:24 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:24.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:24 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:18:24.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:24 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:24.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:24 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:18:24.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:24 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:18:24.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:24 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:24.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:24 smithi115 ceph-mon[41073]: Standby manager daemon x started 2024-09-10T18:18:24.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:24 smithi115 ceph-mon[41073]: from='mgr.? 172.21.15.150:0/750740070' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:18:24.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:24 smithi115 ceph-mon[41073]: from='mgr.? 172.21.15.150:0/750740070' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:18:24.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:24 smithi115 ceph-mon[41073]: from='mgr.? 172.21.15.150:0/750740070' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:18:24.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:24 smithi115 ceph-mon[41073]: from='mgr.? 172.21.15.150:0/750740070' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:18:24.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:24 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:24.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:24 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:24.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:24 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:18:24.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:24 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:24.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:24 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:18:24.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:24 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:24.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:24 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:18:24.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:24 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:18:24.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:24 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:24.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:24 smithi150 ceph-mon[32761]: Standby manager daemon x started 2024-09-10T18:18:24.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:24 smithi150 ceph-mon[32761]: from='mgr.? 172.21.15.150:0/750740070' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:18:24.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:24 smithi150 ceph-mon[32761]: from='mgr.? 172.21.15.150:0/750740070' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:18:24.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:24 smithi150 ceph-mon[32761]: from='mgr.? 172.21.15.150:0/750740070' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:18:24.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:24 smithi150 ceph-mon[32761]: from='mgr.? 172.21.15.150:0/750740070' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:18:25.793 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:25 smithi115 ceph-mon[41073]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:18:25.793 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:25 smithi115 ceph-mon[41073]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:25.793 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:25 smithi115 ceph-mon[41073]: mgrmap e14: y(active, since 62s), standbys: x 2024-09-10T18:18:25.793 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:25 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:18:25.793 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:25 smithi115 ceph-mon[29603]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:18:25.793 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:25 smithi115 ceph-mon[29603]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:25.794 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:25 smithi115 ceph-mon[29603]: mgrmap e14: y(active, since 62s), standbys: x 2024-09-10T18:18:25.794 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:25 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:18:25.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:25 smithi150 ceph-mon[32761]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:18:25.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:25 smithi150 ceph-mon[32761]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:25.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:25 smithi150 ceph-mon[32761]: mgrmap e14: y(active, since 62s), standbys: x 2024-09-10T18:18:25.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:25 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:18:26.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:26 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/3915311989' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "107d9598-28cd-4260-9f5d-a3b7886845c8"}]: dispatch 2024-09-10T18:18:26.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:26 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/3915311989' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "107d9598-28cd-4260-9f5d-a3b7886845c8"}]': finished 2024-09-10T18:18:26.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:26 smithi115 ceph-mon[41073]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T18:18:26.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:26 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:18:26.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:26 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/3915311989' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "107d9598-28cd-4260-9f5d-a3b7886845c8"}]: dispatch 2024-09-10T18:18:26.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:26 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/3915311989' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "107d9598-28cd-4260-9f5d-a3b7886845c8"}]': finished 2024-09-10T18:18:26.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:26 smithi115 ceph-mon[29603]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T18:18:26.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:26 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:18:26.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:26 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/3915311989' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "107d9598-28cd-4260-9f5d-a3b7886845c8"}]: dispatch 2024-09-10T18:18:26.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:26 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/3915311989' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "107d9598-28cd-4260-9f5d-a3b7886845c8"}]': finished 2024-09-10T18:18:26.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:26 smithi150 ceph-mon[32761]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T18:18:26.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:26 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:18:27.772 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:27 smithi115 ceph-mon[41073]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:27.772 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:27 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/3178391910' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:18:27.772 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:27 smithi115 ceph-mon[29603]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:27.773 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:27 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/3178391910' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:18:27.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:27 smithi150 ceph-mon[32761]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:27.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:27 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/3178391910' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:18:29.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:29 smithi150 ceph-mon[32761]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:29.954 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:29 smithi115 ceph-mon[29603]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:29.955 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:29 smithi115 ceph-mon[41073]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:31.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:31 smithi150 ceph-mon[32761]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:31.877 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:31 smithi115 ceph-mon[29603]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:31.877 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:31 smithi115 ceph-mon[41073]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:32.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:32 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T18:18:32.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:32 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:32.871 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:32 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T18:18:32.871 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:32 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:32.872 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:32 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T18:18:32.872 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:32 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:33.625 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:33 smithi115 ceph-mon[29603]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:33.625 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:33 smithi115 ceph-mon[29603]: Deploying daemon osd.0 on smithi115 2024-09-10T18:18:33.626 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:33 smithi115 ceph-mon[41073]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:33.626 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:33 smithi115 ceph-mon[41073]: Deploying daemon osd.0 on smithi115 2024-09-10T18:18:33.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:33 smithi150 ceph-mon[32761]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:33.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:33 smithi150 ceph-mon[32761]: Deploying daemon osd.0 on smithi115 2024-09-10T18:18:35.694 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:35 smithi115 ceph-mon[29603]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:35.694 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:35 smithi115 ceph-mon[41073]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:35.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:35 smithi150 ceph-mon[32761]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:37.779 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:37 smithi115 ceph-mon[41073]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:37.779 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:37 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:18:37.779 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:37 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:37.779 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:37 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:37.779 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:37 smithi115 ceph-mon[29603]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:37.780 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:37 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:18:37.780 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:37 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:37.780 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:37 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:37.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:37 smithi150 ceph-mon[32761]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:37.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:37 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:18:37.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:37 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:37.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:37 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:39.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:39 smithi150 ceph-mon[32761]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:39.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:39 smithi115 ceph-mon[29603]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:39.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:39 smithi115 ceph-mon[41073]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:40.332 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 0 on host 'smithi115' 2024-09-10T18:18:40.646 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:40 smithi115 ceph-mon[41073]: from='osd.0 v2:172.21.15.115:6801/4123495067' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T18:18:40.646 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:40 smithi115 ceph-mon[41073]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T18:18:40.646 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:40 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:40.646 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:40 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:40.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:40 smithi115 ceph-mon[29603]: from='osd.0 v2:172.21.15.115:6801/4123495067' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T18:18:40.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:40 smithi115 ceph-mon[29603]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T18:18:40.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:40 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:40.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:40 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:40.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:40 smithi150 ceph-mon[32761]: from='osd.0 v2:172.21.15.115:6801/4123495067' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T18:18:40.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:40 smithi150 ceph-mon[32761]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T18:18:40.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:40 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:40.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:40 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:40.953 DEBUG:teuthology.orchestra.run.smithi115:osd.0> sudo journalctl -f -n 0 -u ceph-a600e180-6fa0-11ef-bcea-c7b262605968@osd.0.service 2024-09-10T18:18:40.955 INFO:tasks.cephadm:Deploying osd.1 on smithi115 with /dev/vg_nvme/lv_3... 2024-09-10T18:18:40.956 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-10T18:18:41.247 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:18:41.707 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:41 smithi115 ceph-mon[29603]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:41.707 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:41 smithi115 ceph-mon[29603]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T18:18:41.707 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:41 smithi115 ceph-mon[29603]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T18:18:41.708 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:41 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:18:41.708 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:41 smithi115 ceph-mon[29603]: from='osd.0 v2:172.21.15.115:6801/4123495067' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T18:18:41.708 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:41 smithi115 ceph-mon[29603]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T18:18:41.708 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:41 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:41.708 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:41 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:41.708 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:41 smithi115 ceph-mon[41073]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:41.708 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:41 smithi115 ceph-mon[41073]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T18:18:41.708 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:41 smithi115 ceph-mon[41073]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T18:18:41.708 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:41 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:18:41.708 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:41 smithi115 ceph-mon[41073]: from='osd.0 v2:172.21.15.115:6801/4123495067' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T18:18:41.709 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:41 smithi115 ceph-mon[41073]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T18:18:41.709 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:41 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:41.709 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:41 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:41.709 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 18:18:41 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-0[45326]: 2024-09-10T18:18:41.530+0000 7f407ecad640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T18:18:41.709 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 18:18:41 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-0[45326]: 2024-09-10T18:18:41.535+0000 7f407b2e7640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:18:41.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:41 smithi150 ceph-mon[32761]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:41.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:41 smithi150 ceph-mon[32761]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T18:18:41.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:41 smithi150 ceph-mon[32761]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T18:18:41.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:41 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:18:41.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:41 smithi150 ceph-mon[32761]: from='osd.0 v2:172.21.15.115:6801/4123495067' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T18:18:41.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:41 smithi150 ceph-mon[32761]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T18:18:41.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:41 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:41.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:41 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:42.673 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:42 smithi115 ceph-mon[29603]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-10T18:18:42.673 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:42 smithi115 ceph-mon[29603]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T18:18:42.673 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:42 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:18:42.673 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:42 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:18:42.674 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:42 smithi115 ceph-mon[41073]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-10T18:18:42.674 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:42 smithi115 ceph-mon[41073]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T18:18:42.674 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:42 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:18:42.674 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:42 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:18:42.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:42 smithi150 ceph-mon[32761]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-10T18:18:42.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:42 smithi150 ceph-mon[32761]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T18:18:42.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:42 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:18:42.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:42 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:18:43.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:43 smithi115 ceph-mon[29603]: purged_snaps scrub starts 2024-09-10T18:18:43.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:43 smithi115 ceph-mon[29603]: purged_snaps scrub ok 2024-09-10T18:18:43.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:43 smithi115 ceph-mon[29603]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:43.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:43 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:18:43.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:43 smithi115 ceph-mon[29603]: osd.0 v2:172.21.15.115:6801/4123495067 boot 2024-09-10T18:18:43.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:43 smithi115 ceph-mon[29603]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T18:18:43.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:43 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:18:43.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:43 smithi115 ceph-mon[41073]: purged_snaps scrub starts 2024-09-10T18:18:43.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:43 smithi115 ceph-mon[41073]: purged_snaps scrub ok 2024-09-10T18:18:43.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:43 smithi115 ceph-mon[41073]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:43.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:43 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:18:43.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:43 smithi115 ceph-mon[41073]: osd.0 v2:172.21.15.115:6801/4123495067 boot 2024-09-10T18:18:43.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:43 smithi115 ceph-mon[41073]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T18:18:43.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:43 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:18:43.737 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-10T18:18:43.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:43 smithi150 ceph-mon[32761]: purged_snaps scrub starts 2024-09-10T18:18:43.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:43 smithi150 ceph-mon[32761]: purged_snaps scrub ok 2024-09-10T18:18:43.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:43 smithi150 ceph-mon[32761]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:18:43.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:43 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:18:43.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:43 smithi150 ceph-mon[32761]: osd.0 v2:172.21.15.115:6801/4123495067 boot 2024-09-10T18:18:43.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:43 smithi150 ceph-mon[32761]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T18:18:43.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:43 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:18:43.829 INFO:teuthology.orchestra.run.smithi115.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-10T18:18:43.845 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10+0 records in 2024-09-10T18:18:43.845 INFO:teuthology.orchestra.run.smithi115.stderr:10+0 records out 2024-09-10T18:18:43.845 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.014514 s, 722 MB/s 2024-09-10T18:18:43.845 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping successful for: 2024-09-10T18:18:44.658 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:44 smithi115 ceph-mon[29603]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T18:18:44.658 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:44 smithi115 ceph-mon[41073]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T18:18:44.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:44 smithi150 ceph-mon[32761]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T18:18:45.189 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi115:vg_nvme/lv_3 2024-09-10T18:18:45.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:45 smithi115 ceph-mon[29603]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:18:45.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:45 smithi115 ceph-mon[41073]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:18:45.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:45 smithi150 ceph-mon[32761]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:18:45.809 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:18:46.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:46 smithi115 ceph-mon[41073]: Detected new or changed devices on smithi115 2024-09-10T18:18:46.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:46 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:46.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:46 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:46.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:46 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:18:46.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:46 smithi115 ceph-mon[41073]: Adjusting osd_memory_target on smithi115 to 16064M 2024-09-10T18:18:46.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:46 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:46.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:46 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:46.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:46 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:18:46.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:46 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:46.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:46 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:18:46.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:46 smithi115 ceph-mon[41073]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:18:46.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:46 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:46.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:46 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:18:46.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:46 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:46.940 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:46 smithi115 ceph-mon[29603]: Detected new or changed devices on smithi115 2024-09-10T18:18:46.940 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:46 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:46.940 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:46 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:46.940 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:46 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:18:46.940 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:46 smithi115 ceph-mon[29603]: Adjusting osd_memory_target on smithi115 to 16064M 2024-09-10T18:18:46.940 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:46 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:46.940 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:46 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:46.940 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:46 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:18:46.940 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:46 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:46.940 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:46 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:18:46.940 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:46 smithi115 ceph-mon[29603]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:18:46.940 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:46 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:46.941 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:46 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:18:46.941 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:46 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:47.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:46 smithi150 ceph-mon[32761]: Detected new or changed devices on smithi115 2024-09-10T18:18:47.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:46 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:47.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:46 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:47.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:46 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:18:47.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:46 smithi150 ceph-mon[32761]: Adjusting osd_memory_target on smithi115 to 16064M 2024-09-10T18:18:47.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:46 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:47.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:46 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:47.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:46 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:18:47.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:46 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:47.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:46 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:18:47.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:46 smithi150 ceph-mon[32761]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:18:47.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:46 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:47.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:46 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:18:47.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:46 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:18:48.155 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:47 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:18:48.155 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:47 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:18:48.155 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:47 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:48.155 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:47 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:18:48.155 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:47 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:18:48.155 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:47 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:48.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:47 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:18:48.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:47 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:18:48.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:47 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:49.191 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:48 smithi115 ceph-mon[29603]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:18:49.191 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:48 smithi115 ceph-mon[29603]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:18:49.191 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:48 smithi115 ceph-mon[41073]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:18:49.192 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:48 smithi115 ceph-mon[41073]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:18:49.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:48 smithi150 ceph-mon[32761]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:18:49.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:48 smithi150 ceph-mon[32761]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:18:50.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:50 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1682020575' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3cfd69f-7300-4b6e-a57b-33eefed950b1"}]: dispatch 2024-09-10T18:18:50.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:50 smithi150 ceph-mon[32761]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3cfd69f-7300-4b6e-a57b-33eefed950b1"}]: dispatch 2024-09-10T18:18:50.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:50 smithi150 ceph-mon[32761]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3cfd69f-7300-4b6e-a57b-33eefed950b1"}]': finished 2024-09-10T18:18:50.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:50 smithi150 ceph-mon[32761]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T18:18:50.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:50 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:18:50.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:50 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1682020575' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3cfd69f-7300-4b6e-a57b-33eefed950b1"}]: dispatch 2024-09-10T18:18:50.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:50 smithi115 ceph-mon[29603]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3cfd69f-7300-4b6e-a57b-33eefed950b1"}]: dispatch 2024-09-10T18:18:50.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:50 smithi115 ceph-mon[29603]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3cfd69f-7300-4b6e-a57b-33eefed950b1"}]': finished 2024-09-10T18:18:50.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:50 smithi115 ceph-mon[29603]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T18:18:50.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:50 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:18:50.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:50 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1682020575' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3cfd69f-7300-4b6e-a57b-33eefed950b1"}]: dispatch 2024-09-10T18:18:50.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:50 smithi115 ceph-mon[41073]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3cfd69f-7300-4b6e-a57b-33eefed950b1"}]: dispatch 2024-09-10T18:18:50.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:50 smithi115 ceph-mon[41073]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3cfd69f-7300-4b6e-a57b-33eefed950b1"}]': finished 2024-09-10T18:18:50.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:50 smithi115 ceph-mon[41073]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T18:18:50.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:50 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:18:51.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:51 smithi150 ceph-mon[32761]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:18:51.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:51 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/2005743855' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:18:51.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:51 smithi115 ceph-mon[29603]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:18:51.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:51 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/2005743855' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:18:51.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:51 smithi115 ceph-mon[41073]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:18:51.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:51 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/2005743855' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:18:53.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:53 smithi150 ceph-mon[32761]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:18:53.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:53 smithi115 ceph-mon[29603]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:18:53.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:53 smithi115 ceph-mon[41073]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:18:55.285 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:55 smithi115 ceph-mon[41073]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:18:55.285 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:55 smithi115 ceph-mon[29603]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:18:55.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:55 smithi150 ceph-mon[32761]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:18:57.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:57 smithi150 ceph-mon[32761]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:18:57.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:57 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T18:18:57.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:57 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:57.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:57 smithi150 ceph-mon[32761]: Deploying daemon osd.1 on smithi115 2024-09-10T18:18:57.334 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:57 smithi115 ceph-mon[29603]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:18:57.334 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:57 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T18:18:57.335 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:57 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:57.335 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:57 smithi115 ceph-mon[29603]: Deploying daemon osd.1 on smithi115 2024-09-10T18:18:57.335 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:57 smithi115 ceph-mon[41073]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:18:57.335 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:57 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T18:18:57.335 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:57 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:57.335 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:57 smithi115 ceph-mon[41073]: Deploying daemon osd.1 on smithi115 2024-09-10T18:18:59.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:18:59 smithi150 ceph-mon[32761]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:18:59.416 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:18:59 smithi115 ceph-mon[29603]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:18:59.417 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:18:59 smithi115 ceph-mon[41073]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:19:01.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:01 smithi150 ceph-mon[32761]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:19:01.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:01 smithi115 ceph-mon[29603]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:19:01.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:01 smithi115 ceph-mon[41073]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:19:02.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:02 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:19:02.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:02 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:02.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:02 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:02.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:02 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:19:02.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:02 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:02.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:02 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:02.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:02 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:19:02.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:02 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:02.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:02 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:03.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:03 smithi115 ceph-mon[29603]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:19:03.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:03 smithi115 ceph-mon[41073]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:19:03.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:03 smithi150 ceph-mon[32761]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:19:04.516 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 1 on host 'smithi115' 2024-09-10T18:19:05.137 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:04 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:05.138 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:04 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:05.138 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:04 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:05.138 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:04 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:19:05.138 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:04 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:05.138 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:04 smithi115 ceph-mon[41073]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:19:05.138 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:04 smithi115 ceph-mon[41073]: from='osd.1 v2:172.21.15.115:6805/2506968391' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T18:19:05.138 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:04 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:19:05.138 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:04 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:05.138 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:04 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:05.139 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:04 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:05.139 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:04 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:05.139 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:04 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:05.139 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:04 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:19:05.139 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:04 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:05.139 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:04 smithi115 ceph-mon[29603]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:19:05.139 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:04 smithi115 ceph-mon[29603]: from='osd.1 v2:172.21.15.115:6805/2506968391' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T18:19:05.139 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:04 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:19:05.139 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:04 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:05.139 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:04 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:05.162 DEBUG:teuthology.orchestra.run.smithi115:osd.1> sudo journalctl -f -n 0 -u ceph-a600e180-6fa0-11ef-bcea-c7b262605968@osd.1.service 2024-09-10T18:19:05.165 INFO:tasks.cephadm:Deploying osd.2 on smithi115 with /dev/vg_nvme/lv_2... 2024-09-10T18:19:05.165 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-10T18:19:05.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:04 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:05.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:04 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:05.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:04 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:05.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:04 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:19:05.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:04 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:05.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:04 smithi150 ceph-mon[32761]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:19:05.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:04 smithi150 ceph-mon[32761]: from='osd.1 v2:172.21.15.115:6805/2506968391' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T18:19:05.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:04 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:19:05.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:04 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:05.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:04 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:05.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:05 smithi115 ceph-mon[29603]: from='osd.1 v2:172.21.15.115:6805/2506968391' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T18:19:05.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:05 smithi115 ceph-mon[29603]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T18:19:05.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:05 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:19:05.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:05 smithi115 ceph-mon[29603]: from='osd.1 v2:172.21.15.115:6805/2506968391' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T18:19:05.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:05 smithi115 ceph-mon[41073]: from='osd.1 v2:172.21.15.115:6805/2506968391' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T18:19:05.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:05 smithi115 ceph-mon[41073]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T18:19:05.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:05 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:19:05.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:05 smithi115 ceph-mon[41073]: from='osd.1 v2:172.21.15.115:6805/2506968391' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T18:19:05.986 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 18:19:05 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-1[51767]: 2024-09-10T18:19:05.842+0000 7fef7b88a640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T18:19:05.986 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 18:19:05 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-1[51767]: 2024-09-10T18:19:05.853+0000 7fef766c1640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:19:06.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:05 smithi150 ceph-mon[32761]: from='osd.1 v2:172.21.15.115:6805/2506968391' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T18:19:06.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:05 smithi150 ceph-mon[32761]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T18:19:06.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:05 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:19:06.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:05 smithi150 ceph-mon[32761]: from='osd.1 v2:172.21.15.115:6805/2506968391' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T18:19:06.922 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:19:06.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:06 smithi115 ceph-mon[29603]: from='osd.1 v2:172.21.15.115:6805/2506968391' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-10T18:19:06.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:06 smithi115 ceph-mon[29603]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T18:19:06.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:06 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:19:06.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:06 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:19:06.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:06 smithi115 ceph-mon[29603]: pgmap v54: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:19:06.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:06 smithi115 ceph-mon[41073]: from='osd.1 v2:172.21.15.115:6805/2506968391' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-10T18:19:06.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:06 smithi115 ceph-mon[41073]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T18:19:06.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:06 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:19:06.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:06 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:19:06.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:06 smithi115 ceph-mon[41073]: pgmap v54: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:19:07.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:06 smithi150 ceph-mon[32761]: from='osd.1 v2:172.21.15.115:6805/2506968391' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-10T18:19:07.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:06 smithi150 ceph-mon[32761]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T18:19:07.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:06 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:19:07.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:06 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:19:07.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:06 smithi150 ceph-mon[32761]: pgmap v54: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:19:07.956 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:07 smithi115 ceph-mon[41073]: purged_snaps scrub starts 2024-09-10T18:19:07.956 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:07 smithi115 ceph-mon[41073]: purged_snaps scrub ok 2024-09-10T18:19:07.956 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:07 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:19:07.956 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:07 smithi115 ceph-mon[41073]: osd.1 v2:172.21.15.115:6805/2506968391 boot 2024-09-10T18:19:07.956 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:07 smithi115 ceph-mon[41073]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T18:19:07.956 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:07 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:19:07.957 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:07 smithi115 ceph-mon[29603]: purged_snaps scrub starts 2024-09-10T18:19:07.957 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:07 smithi115 ceph-mon[29603]: purged_snaps scrub ok 2024-09-10T18:19:07.957 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:07 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:19:07.957 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:07 smithi115 ceph-mon[29603]: osd.1 v2:172.21.15.115:6805/2506968391 boot 2024-09-10T18:19:07.957 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:07 smithi115 ceph-mon[29603]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T18:19:07.957 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:07 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:19:08.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:07 smithi150 ceph-mon[32761]: purged_snaps scrub starts 2024-09-10T18:19:08.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:07 smithi150 ceph-mon[32761]: purged_snaps scrub ok 2024-09-10T18:19:08.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:07 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:19:08.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:07 smithi150 ceph-mon[32761]: osd.1 v2:172.21.15.115:6805/2506968391 boot 2024-09-10T18:19:08.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:07 smithi150 ceph-mon[32761]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T18:19:08.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:07 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:19:09.094 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:08 smithi115 ceph-mon[29603]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T18:19:09.094 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:08 smithi115 ceph-mon[29603]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:09.095 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:08 smithi115 ceph-mon[29603]: Detected new or changed devices on smithi115 2024-09-10T18:19:09.095 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:08 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:09.095 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:08 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:09.095 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:08 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:19:09.095 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:08 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:19:09.095 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:08 smithi115 ceph-mon[29603]: Adjusting osd_memory_target on smithi115 to 8032M 2024-09-10T18:19:09.095 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:08 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:09.095 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:08 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:09.095 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:08 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:19:09.095 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:08 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:09.096 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:08 smithi115 ceph-mon[41073]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T18:19:09.096 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:08 smithi115 ceph-mon[41073]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:09.096 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:08 smithi115 ceph-mon[41073]: Detected new or changed devices on smithi115 2024-09-10T18:19:09.096 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:08 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:09.096 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:08 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:09.096 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:08 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:19:09.096 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:08 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:19:09.096 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:08 smithi115 ceph-mon[41073]: Adjusting osd_memory_target on smithi115 to 8032M 2024-09-10T18:19:09.096 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:08 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:09.096 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:08 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:09.096 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:08 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:19:09.096 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:08 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:09.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:08 smithi150 ceph-mon[32761]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T18:19:09.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:08 smithi150 ceph-mon[32761]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:09.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:08 smithi150 ceph-mon[32761]: Detected new or changed devices on smithi115 2024-09-10T18:19:09.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:08 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:09.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:08 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:09.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:08 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:19:09.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:08 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:19:09.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:08 smithi150 ceph-mon[32761]: Adjusting osd_memory_target on smithi115 to 8032M 2024-09-10T18:19:09.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:08 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:09.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:08 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:09.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:08 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:19:09.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:08 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:09.440 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-10T18:19:09.542 INFO:teuthology.orchestra.run.smithi115.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-10T18:19:09.556 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10+0 records in 2024-09-10T18:19:09.556 INFO:teuthology.orchestra.run.smithi115.stderr:10+0 records out 2024-09-10T18:19:09.556 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126327 s, 830 MB/s 2024-09-10T18:19:09.557 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping successful for: 2024-09-10T18:19:10.181 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi115:vg_nvme/lv_2 2024-09-10T18:19:10.422 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:19:11.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:11 smithi115 ceph-mon[41073]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:11.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:11 smithi115 ceph-mon[29603]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:11.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:11 smithi150 ceph-mon[32761]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:13.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:13 smithi115 ceph-mon[41073]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:13.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:13 smithi115 ceph-mon[41073]: from='client.14271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:19:13.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:13 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:19:13.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:13 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:19:13.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:13 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:13.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:13 smithi115 ceph-mon[29603]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:13.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:13 smithi115 ceph-mon[29603]: from='client.14271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:19:13.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:13 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:19:13.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:13 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:19:13.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:13 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:13.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:13 smithi150 ceph-mon[32761]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:13.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:13 smithi150 ceph-mon[32761]: from='client.14271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:19:13.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:13 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:19:13.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:13 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:19:13.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:13 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:15.141 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:15 smithi115 ceph-mon[29603]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:15.141 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:15 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/3002847910' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9abe6878-817d-4c81-b4b8-db61147c2f94"}]: dispatch 2024-09-10T18:19:15.141 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:15 smithi115 ceph-mon[29603]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9abe6878-817d-4c81-b4b8-db61147c2f94"}]: dispatch 2024-09-10T18:19:15.142 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:15 smithi115 ceph-mon[29603]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9abe6878-817d-4c81-b4b8-db61147c2f94"}]': finished 2024-09-10T18:19:15.142 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:15 smithi115 ceph-mon[29603]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T18:19:15.142 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:15 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:19:15.142 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:15 smithi115 ceph-mon[41073]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:15.142 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:15 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/3002847910' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9abe6878-817d-4c81-b4b8-db61147c2f94"}]: dispatch 2024-09-10T18:19:15.142 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:15 smithi115 ceph-mon[41073]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9abe6878-817d-4c81-b4b8-db61147c2f94"}]: dispatch 2024-09-10T18:19:15.142 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:15 smithi115 ceph-mon[41073]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9abe6878-817d-4c81-b4b8-db61147c2f94"}]': finished 2024-09-10T18:19:15.142 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:15 smithi115 ceph-mon[41073]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T18:19:15.142 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:15 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:19:15.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:15 smithi150 ceph-mon[32761]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:15.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:15 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/3002847910' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9abe6878-817d-4c81-b4b8-db61147c2f94"}]: dispatch 2024-09-10T18:19:15.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:15 smithi150 ceph-mon[32761]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9abe6878-817d-4c81-b4b8-db61147c2f94"}]: dispatch 2024-09-10T18:19:15.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:15 smithi150 ceph-mon[32761]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9abe6878-817d-4c81-b4b8-db61147c2f94"}]': finished 2024-09-10T18:19:15.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:15 smithi150 ceph-mon[32761]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T18:19:15.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:15 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:19:16.386 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:16 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1685212627' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:19:16.387 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:16 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1685212627' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:19:16.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:16 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1685212627' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:19:17.328 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:17 smithi115 ceph-mon[41073]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:17.328 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:17 smithi115 ceph-mon[29603]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:17.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:17 smithi150 ceph-mon[32761]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:19.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:19 smithi115 ceph-mon[29603]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:19.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:19 smithi115 ceph-mon[41073]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:19.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:19 smithi150 ceph-mon[32761]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:21.345 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:21 smithi115 ceph-mon[41073]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:21.345 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:21 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T18:19:21.345 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:21 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:21.345 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:21 smithi115 ceph-mon[29603]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:21.345 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:21 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T18:19:21.346 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:21 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:21.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:21 smithi150 ceph-mon[32761]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:21.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:21 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T18:19:21.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:21 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:22.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:22 smithi115 ceph-mon[29603]: Deploying daemon osd.2 on smithi115 2024-09-10T18:19:22.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:22 smithi115 ceph-mon[41073]: Deploying daemon osd.2 on smithi115 2024-09-10T18:19:22.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:22 smithi150 ceph-mon[32761]: Deploying daemon osd.2 on smithi115 2024-09-10T18:19:23.374 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:23 smithi115 ceph-mon[41073]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:23.386 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:23 smithi115 ceph-mon[29603]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:23.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:23 smithi150 ceph-mon[32761]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:25.156 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:25 smithi115 ceph-mon[41073]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:25.157 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:25 smithi115 ceph-mon[29603]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:25.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:25 smithi150 ceph-mon[32761]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:26.334 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:26 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:19:26.334 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:26 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:26.334 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:26 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:26.334 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:26 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:19:26.334 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:26 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:26.335 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:26 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:26.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:26 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:19:26.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:26 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:26.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:26 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:27.484 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:27 smithi115 ceph-mon[29603]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:27.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:27 smithi115 ceph-mon[41073]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:27.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:27 smithi150 ceph-mon[32761]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:29.171 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 2 on host 'smithi115' 2024-09-10T18:19:29.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:29 smithi150 ceph-mon[32761]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:29.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:29 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:29.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:29 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:29.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:29 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:29.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:29 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:19:29.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:29 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:29.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:29 smithi150 ceph-mon[32761]: from='osd.2 v2:172.21.15.115:6809/343149052' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:19:29.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:29 smithi150 ceph-mon[32761]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:19:29.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:29 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:19:29.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:29 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:29.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:29 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:29.830 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:29 smithi115 ceph-mon[29603]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:29.830 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:29 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:29.830 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:29 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:29.830 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:29 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:29.830 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:29 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:19:29.830 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:29 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:29.830 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:29 smithi115 ceph-mon[29603]: from='osd.2 v2:172.21.15.115:6809/343149052' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:19:29.830 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:29 smithi115 ceph-mon[29603]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:19:29.830 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:29 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:19:29.830 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:29 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:29.831 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:29 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:29.831 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:29 smithi115 ceph-mon[41073]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:29.831 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:29 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:29.831 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:29 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:29.831 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:29 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:29.831 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:29 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:19:29.831 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:29 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:29.831 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:29 smithi115 ceph-mon[41073]: from='osd.2 v2:172.21.15.115:6809/343149052' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:19:29.831 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:29 smithi115 ceph-mon[41073]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:19:29.831 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:29 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:19:29.831 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:29 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:29.831 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:29 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:29.854 DEBUG:teuthology.orchestra.run.smithi115:osd.2> sudo journalctl -f -n 0 -u ceph-a600e180-6fa0-11ef-bcea-c7b262605968@osd.2.service 2024-09-10T18:19:29.856 INFO:tasks.cephadm:Deploying osd.3 on smithi115 with /dev/vg_nvme/lv_1... 2024-09-10T18:19:29.857 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-10T18:19:30.649 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:19:30.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:30 smithi150 ceph-mon[32761]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T18:19:30.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:30 smithi150 ceph-mon[32761]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T18:19:30.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:30 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:19:30.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:30 smithi150 ceph-mon[32761]: from='osd.2 v2:172.21.15.115:6809/343149052' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T18:19:30.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:30 smithi150 ceph-mon[32761]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T18:19:30.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:30 smithi115 ceph-mon[29603]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T18:19:30.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:30 smithi115 ceph-mon[29603]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T18:19:30.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:30 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:19:30.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:30 smithi115 ceph-mon[29603]: from='osd.2 v2:172.21.15.115:6809/343149052' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T18:19:30.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:30 smithi115 ceph-mon[29603]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T18:19:30.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:30 smithi115 ceph-mon[41073]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T18:19:30.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:30 smithi115 ceph-mon[41073]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T18:19:30.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:30 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:19:30.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:30 smithi115 ceph-mon[41073]: from='osd.2 v2:172.21.15.115:6809/343149052' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T18:19:30.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:30 smithi115 ceph-mon[41073]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T18:19:30.986 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 18:19:30 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-2[58201]: 2024-09-10T18:19:30.543+0000 7f4039ca9640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T18:19:30.986 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 18:19:30 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-2[58201]: 2024-09-10T18:19:30.546+0000 7f4032adc640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:19:31.581 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:31 smithi115 ceph-mon[41073]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:31.581 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:31 smithi115 ceph-mon[41073]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-10T18:19:31.581 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:31 smithi115 ceph-mon[41073]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T18:19:31.581 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:31 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:19:31.581 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:31 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:19:31.581 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:31 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:19:31.582 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:31 smithi115 ceph-mon[29603]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:31.582 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:31 smithi115 ceph-mon[29603]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-10T18:19:31.582 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:31 smithi115 ceph-mon[29603]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T18:19:31.582 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:31 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:19:31.582 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:31 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:19:31.582 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:31 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:19:31.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:31 smithi150 ceph-mon[32761]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:31.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:31 smithi150 ceph-mon[32761]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-10T18:19:31.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:31 smithi150 ceph-mon[32761]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T18:19:31.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:31 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:19:31.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:31 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:19:31.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:31 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:19:32.952 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:32 smithi115 ceph-mon[29603]: purged_snaps scrub starts 2024-09-10T18:19:32.952 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:32 smithi115 ceph-mon[29603]: purged_snaps scrub ok 2024-09-10T18:19:32.953 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:32 smithi115 ceph-mon[29603]: osd.2 v2:172.21.15.115:6809/343149052 boot 2024-09-10T18:19:32.953 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:32 smithi115 ceph-mon[29603]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T18:19:32.953 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:32 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:19:32.953 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:32 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-10T18:19:32.953 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:32 smithi115 ceph-mon[41073]: purged_snaps scrub starts 2024-09-10T18:19:32.953 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:32 smithi115 ceph-mon[41073]: purged_snaps scrub ok 2024-09-10T18:19:32.953 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:32 smithi115 ceph-mon[41073]: osd.2 v2:172.21.15.115:6809/343149052 boot 2024-09-10T18:19:32.953 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:32 smithi115 ceph-mon[41073]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T18:19:32.953 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:32 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:19:32.953 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:32 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-10T18:19:33.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:32 smithi150 ceph-mon[32761]: purged_snaps scrub starts 2024-09-10T18:19:33.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:32 smithi150 ceph-mon[32761]: purged_snaps scrub ok 2024-09-10T18:19:33.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:32 smithi150 ceph-mon[32761]: osd.2 v2:172.21.15.115:6809/343149052 boot 2024-09-10T18:19:33.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:32 smithi150 ceph-mon[32761]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T18:19:33.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:32 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:19:33.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:32 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-10T18:19:33.453 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-10T18:19:33.546 INFO:teuthology.orchestra.run.smithi115.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-10T18:19:33.565 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10+0 records in 2024-09-10T18:19:33.565 INFO:teuthology.orchestra.run.smithi115.stderr:10+0 records out 2024-09-10T18:19:33.565 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0171752 s, 611 MB/s 2024-09-10T18:19:33.566 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping successful for: 2024-09-10T18:19:33.778 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:33 smithi115 ceph-mon[41073]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:33.778 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:33 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-10T18:19:33.778 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:33 smithi115 ceph-mon[41073]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T18:19:33.778 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:33 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:19:33.779 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:33 smithi115 ceph-mon[29603]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:33.779 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:33 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-10T18:19:33.779 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:33 smithi115 ceph-mon[29603]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T18:19:33.779 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:33 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:19:34.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:33 smithi150 ceph-mon[32761]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:19:34.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:33 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-10T18:19:34.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:33 smithi150 ceph-mon[32761]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T18:19:34.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:33 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:19:34.449 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi115:vg_nvme/lv_1 2024-09-10T18:19:34.638 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 18:19:34 smithi115 sudo[62033]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:19:34.638 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 18:19:34 smithi115 sudo[62033]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:19:34.638 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 18:19:34 smithi115 sudo[62033]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:19:34.638 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 18:19:34 smithi115 sudo[62033]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:19:34.639 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:34 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:19:34.639 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:34 smithi115 ceph-mon[41073]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T18:19:34.639 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:34 smithi115 ceph-mon[41073]: Detected new or changed devices on smithi115 2024-09-10T18:19:34.639 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:34 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:34.639 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:34 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:34.639 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:34 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:19:34.639 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:34 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:19:34.639 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:34 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:19:34.639 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:34 smithi115 ceph-mon[41073]: Adjusting osd_memory_target on smithi115 to 5354M 2024-09-10T18:19:34.639 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:34 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:34.639 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:34 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:34.639 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:34 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:19:34.639 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:34 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:34.640 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:34 smithi115 ceph-mon[41073]: pgmap v76: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:19:34.640 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:34 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:19:34.640 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:34 smithi115 ceph-mon[29603]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T18:19:34.640 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:34 smithi115 ceph-mon[29603]: Detected new or changed devices on smithi115 2024-09-10T18:19:34.640 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:34 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:34.640 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:34 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:34.640 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:34 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:19:34.640 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:34 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:19:34.640 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:34 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:19:34.640 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:34 smithi115 ceph-mon[29603]: Adjusting osd_memory_target on smithi115 to 5354M 2024-09-10T18:19:34.640 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:34 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:34.641 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:34 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:34.641 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:34 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:19:34.641 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:34 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:34.641 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:34 smithi115 ceph-mon[29603]: pgmap v76: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:19:34.699 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:19:34.911 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 18:19:34 smithi115 sudo[62067]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:19:34.911 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 18:19:34 smithi115 sudo[62067]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:19:34.911 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 18:19:34 smithi115 sudo[62067]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:19:34.911 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 18:19:34 smithi115 sudo[62067]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:19:35.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:34 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:19:35.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:34 smithi150 ceph-mon[32761]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T18:19:35.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:34 smithi150 ceph-mon[32761]: Detected new or changed devices on smithi115 2024-09-10T18:19:35.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:34 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:35.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:34 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:35.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:34 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:19:35.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:34 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:19:35.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:34 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:19:35.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:34 smithi150 ceph-mon[32761]: Adjusting osd_memory_target on smithi115 to 5354M 2024-09-10T18:19:35.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:34 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:35.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:34 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:35.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:34 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:19:35.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:34 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:35.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:34 smithi150 ceph-mon[32761]: pgmap v76: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:19:35.176 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 18:19:34 smithi115 sudo[62113]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:19:35.176 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 18:19:34 smithi115 sudo[62113]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:19:35.176 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 18:19:34 smithi115 sudo[62113]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:19:35.176 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 18:19:34 smithi115 sudo[62113]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:19:35.439 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 18:19:35 smithi115 sudo[62149]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:19:35.440 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 18:19:35 smithi115 sudo[62149]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:19:35.440 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 18:19:35 smithi115 sudo[62149]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:19:35.440 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 18:19:35 smithi115 sudo[62149]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:19:35.440 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 18:19:35 smithi115 sudo[62156]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:19:35.440 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 18:19:35 smithi115 sudo[62156]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:19:35.440 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 18:19:35 smithi115 sudo[62156]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:19:35.440 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 18:19:35 smithi115 sudo[62156]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:19:35.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:35 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:35.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:35 smithi115 ceph-mon[29603]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T18:19:35.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:35 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:35.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:35 smithi115 ceph-mon[41073]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T18:19:35.735 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 18:19:35 smithi115 sudo[62169]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:19:35.735 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 18:19:35 smithi115 sudo[62169]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:19:35.735 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 18:19:35 smithi115 sudo[62169]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:19:35.735 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 18:19:35 smithi115 sudo[62169]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:19:36.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:35 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:36.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:35 smithi150 ceph-mon[32761]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T18:19:36.194 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:35 smithi115 sudo[62172]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:19:36.194 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:35 smithi115 sudo[62172]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:19:36.194 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:35 smithi115 sudo[62172]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:19:36.194 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:35 smithi115 sudo[62172]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:19:36.458 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:36 smithi115 sudo[62196]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:19:36.458 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:36 smithi115 sudo[62196]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:19:36.459 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:36 smithi115 sudo[62196]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:19:36.459 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:36 smithi115 sudo[62196]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:19:36.694 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:36 smithi150 sudo[35489]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:19:36.694 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:36 smithi150 sudo[35489]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:19:36.694 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:36 smithi150 sudo[35489]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:19:36.694 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:36 smithi150 sudo[35489]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:19:36.977 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:36 smithi115 ceph-mon[41073]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:19:36.978 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:36 smithi115 ceph-mon[41073]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:19:36.978 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:36 smithi115 ceph-mon[41073]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:19:36.978 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:36 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:19:36.978 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:36 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:19:36.978 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:36 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:19:36.978 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:36 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:19:36.978 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:36 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:19:36.978 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:36 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:19:36.978 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:36 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:19:36.978 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:36 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:19:36.978 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:36 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:36.979 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:36 smithi115 ceph-mon[29603]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:19:36.979 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:36 smithi115 ceph-mon[29603]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:19:36.979 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:36 smithi115 ceph-mon[29603]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:19:36.979 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:36 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:19:36.979 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:36 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:19:36.979 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:36 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:19:36.979 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:36 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:19:36.979 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:36 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:19:36.979 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:36 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:19:36.979 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:36 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:19:36.979 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:36 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:19:36.979 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:36 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:37.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:36 smithi150 sudo[35492]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:19:37.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:36 smithi150 sudo[35492]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:19:37.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:36 smithi150 sudo[35492]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:19:37.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:36 smithi150 sudo[35492]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:19:37.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:36 smithi150 ceph-mon[32761]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:19:37.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:36 smithi150 ceph-mon[32761]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:19:37.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:36 smithi150 ceph-mon[32761]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:19:37.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:36 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:19:37.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:36 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:19:37.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:36 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:19:37.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:36 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:19:37.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:36 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:19:37.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:36 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:19:37.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:36 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:19:37.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:36 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:19:37.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:36 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:37.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:36 smithi115 sudo[62367]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:19:37.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:36 smithi115 sudo[62367]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:19:37.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:36 smithi115 sudo[62367]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:19:37.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:36 smithi115 sudo[62367]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:19:37.713 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:37 smithi115 sudo[62389]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:19:37.713 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:37 smithi115 sudo[62389]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:19:37.713 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:37 smithi115 sudo[62389]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:19:37.713 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:37 smithi115 sudo[62389]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:19:37.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:37 smithi115 ceph-mon[29603]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:19:37.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:37 smithi115 ceph-mon[29603]: from='client.14307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:19:37.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:37 smithi115 ceph-mon[29603]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:19:37.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:37 smithi115 ceph-mon[29603]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-10T18:19:37.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:37 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:37.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:37 smithi115 ceph-mon[29603]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:19:37.986 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:37 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:19:37.986 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:37 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:19:37.986 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:37 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:19:37.986 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:37 smithi115 ceph-mon[29603]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:19:37.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:37 smithi115 ceph-mon[41073]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:19:37.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:37 smithi115 ceph-mon[41073]: from='client.14307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:19:37.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:37 smithi115 ceph-mon[41073]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:19:37.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:37 smithi115 ceph-mon[41073]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-10T18:19:37.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:37 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:37.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:37 smithi115 ceph-mon[41073]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:19:37.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:37 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:19:37.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:37 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:19:37.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:37 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:19:37.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:37 smithi115 ceph-mon[41073]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:19:38.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:37 smithi150 ceph-mon[32761]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:19:38.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:37 smithi150 ceph-mon[32761]: from='client.14307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:19:38.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:37 smithi150 ceph-mon[32761]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:19:38.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:37 smithi150 ceph-mon[32761]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-10T18:19:38.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:37 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:38.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:37 smithi150 ceph-mon[32761]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:19:38.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:37 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:19:38.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:37 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:19:38.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:37 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:19:38.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:37 smithi150 ceph-mon[32761]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:19:38.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:38 smithi115 ceph-mon[29603]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:19:38.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:38 smithi115 ceph-mon[41073]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:19:39.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:38 smithi150 ceph-mon[32761]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:19:39.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:39 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/4254491940' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4fefb92a-fc8d-4a28-9c89-1b7e214c8e0e"}]: dispatch 2024-09-10T18:19:39.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:39 smithi115 ceph-mon[29603]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4fefb92a-fc8d-4a28-9c89-1b7e214c8e0e"}]: dispatch 2024-09-10T18:19:39.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:39 smithi115 ceph-mon[29603]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4fefb92a-fc8d-4a28-9c89-1b7e214c8e0e"}]': finished 2024-09-10T18:19:39.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:39 smithi115 ceph-mon[29603]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T18:19:39.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:39 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:19:39.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:39 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/4254491940' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4fefb92a-fc8d-4a28-9c89-1b7e214c8e0e"}]: dispatch 2024-09-10T18:19:39.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:39 smithi115 ceph-mon[41073]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4fefb92a-fc8d-4a28-9c89-1b7e214c8e0e"}]: dispatch 2024-09-10T18:19:39.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:39 smithi115 ceph-mon[41073]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4fefb92a-fc8d-4a28-9c89-1b7e214c8e0e"}]': finished 2024-09-10T18:19:39.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:39 smithi115 ceph-mon[41073]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T18:19:39.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:39 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:19:40.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:39 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/4254491940' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4fefb92a-fc8d-4a28-9c89-1b7e214c8e0e"}]: dispatch 2024-09-10T18:19:40.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:39 smithi150 ceph-mon[32761]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4fefb92a-fc8d-4a28-9c89-1b7e214c8e0e"}]: dispatch 2024-09-10T18:19:40.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:39 smithi150 ceph-mon[32761]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4fefb92a-fc8d-4a28-9c89-1b7e214c8e0e"}]': finished 2024-09-10T18:19:40.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:39 smithi150 ceph-mon[32761]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T18:19:40.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:39 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:19:40.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:40 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/3361137305' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:19:40.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:40 smithi115 ceph-mon[41073]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:19:40.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:40 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/3361137305' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:19:40.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:40 smithi115 ceph-mon[29603]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:19:41.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:40 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/3361137305' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:19:41.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:40 smithi150 ceph-mon[32761]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:19:43.335 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:43 smithi115 ceph-mon[41073]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:19:43.335 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:43 smithi115 ceph-mon[29603]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:19:43.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:43 smithi150 ceph-mon[32761]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:19:45.428 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:45 smithi115 ceph-mon[41073]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:19:45.429 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:45 smithi115 ceph-mon[29603]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:19:45.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:45 smithi150 ceph-mon[32761]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:19:46.123 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:46 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T18:19:46.123 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:46 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:46.123 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:46 smithi115 ceph-mon[29603]: Deploying daemon osd.3 on smithi115 2024-09-10T18:19:46.123 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:46 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T18:19:46.124 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:46 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:46.124 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:46 smithi115 ceph-mon[41073]: Deploying daemon osd.3 on smithi115 2024-09-10T18:19:46.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:46 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T18:19:46.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:46 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:46.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:46 smithi150 ceph-mon[32761]: Deploying daemon osd.3 on smithi115 2024-09-10T18:19:47.155 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:47 smithi115 ceph-mon[41073]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:19:47.155 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:47 smithi115 ceph-mon[29603]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:19:47.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:47 smithi150 ceph-mon[32761]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:19:49.324 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:49 smithi115 ceph-mon[29603]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:19:49.324 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:49 smithi115 ceph-mon[41073]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:19:49.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:49 smithi150 ceph-mon[32761]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:19:51.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:51 smithi115 ceph-mon[29603]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:19:51.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:51 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:19:51.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:51 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:51.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:51 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:51.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:51 smithi115 ceph-mon[41073]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:19:51.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:51 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:19:51.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:51 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:51.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:51 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:51.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:51 smithi150 ceph-mon[32761]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:19:51.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:51 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:19:51.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:51 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:51.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:51 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:53.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:53 smithi115 ceph-mon[29603]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:19:53.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:53 smithi115 ceph-mon[41073]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:19:53.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:53 smithi150 ceph-mon[32761]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:19:54.011 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 3 on host 'smithi115' 2024-09-10T18:19:54.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:54 smithi115 ceph-mon[29603]: from='osd.3 v2:172.21.15.115:6813/3162056893' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T18:19:54.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:54 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:54.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:54 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:54.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:54 smithi115 ceph-mon[41073]: from='osd.3 v2:172.21.15.115:6813/3162056893' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T18:19:54.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:54 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:54.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:54 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:54.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:54 smithi150 ceph-mon[32761]: from='osd.3 v2:172.21.15.115:6813/3162056893' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T18:19:54.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:54 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:54.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:54 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:54.614 DEBUG:teuthology.orchestra.run.smithi115:osd.3> sudo journalctl -f -n 0 -u ceph-a600e180-6fa0-11ef-bcea-c7b262605968@osd.3.service 2024-09-10T18:19:54.616 INFO:tasks.cephadm:Deploying osd.4 on smithi150 with /dev/vg_nvme/lv_4... 2024-09-10T18:19:54.616 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-10T18:19:54.867 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:19:55.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:55 smithi150 ceph-mon[32761]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:19:55.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:55 smithi150 ceph-mon[32761]: from='osd.3 v2:172.21.15.115:6813/3162056893' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T18:19:55.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:55 smithi150 ceph-mon[32761]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T18:19:55.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:55 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:19:55.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:55 smithi150 ceph-mon[32761]: from='osd.3 v2:172.21.15.115:6813/3162056893' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T18:19:55.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:55 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:55.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:55 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:55.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:55 smithi115 ceph-mon[41073]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:19:55.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:55 smithi115 ceph-mon[41073]: from='osd.3 v2:172.21.15.115:6813/3162056893' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T18:19:55.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:55 smithi115 ceph-mon[41073]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T18:19:55.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:55 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:19:55.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:55 smithi115 ceph-mon[41073]: from='osd.3 v2:172.21.15.115:6813/3162056893' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T18:19:55.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:55 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:55.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:55 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:55.433 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:55 smithi115 ceph-mon[29603]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:19:55.433 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:55 smithi115 ceph-mon[29603]: from='osd.3 v2:172.21.15.115:6813/3162056893' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T18:19:55.433 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:55 smithi115 ceph-mon[29603]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T18:19:55.433 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:55 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:19:55.433 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:55 smithi115 ceph-mon[29603]: from='osd.3 v2:172.21.15.115:6813/3162056893' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T18:19:55.433 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:55 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:55.433 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:55 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:55.433 INFO:journalctl@ceph.osd.3.smithi115.stdout:Sep 10 18:19:55 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-3[64304]: 2024-09-10T18:19:55.082+0000 7fad7187d640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T18:19:55.433 INFO:journalctl@ceph.osd.3.smithi115.stdout:Sep 10 18:19:55 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-3[64304]: 2024-09-10T18:19:55.087+0000 7fad6ceb5640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:19:56.197 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:56 smithi150 ceph-mon[32761]: from='osd.3 v2:172.21.15.115:6813/3162056893' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-10T18:19:56.198 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:56 smithi150 ceph-mon[32761]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T18:19:56.198 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:56 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:19:56.198 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:56 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:19:56.198 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:56 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:19:56.462 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:56 smithi115 ceph-mon[29603]: from='osd.3 v2:172.21.15.115:6813/3162056893' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-10T18:19:56.462 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:56 smithi115 ceph-mon[29603]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T18:19:56.462 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:56 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:19:56.462 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:56 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:19:56.462 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:56 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:19:56.462 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:56 smithi115 ceph-mon[41073]: from='osd.3 v2:172.21.15.115:6813/3162056893' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-10T18:19:56.462 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:56 smithi115 ceph-mon[41073]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T18:19:56.463 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:56 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:19:56.463 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:56 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:19:56.463 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:56 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:19:56.537 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-10T18:19:56.625 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-10T18:19:56.640 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-09-10T18:19:56.641 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-09-10T18:19:56.641 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0132716 s, 790 MB/s 2024-09-10T18:19:56.641 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-09-10T18:19:57.196 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:57 smithi150 ceph-mon[32761]: purged_snaps scrub starts 2024-09-10T18:19:57.196 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:57 smithi150 ceph-mon[32761]: purged_snaps scrub ok 2024-09-10T18:19:57.196 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:57 smithi150 ceph-mon[32761]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:19:57.196 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:57 smithi150 ceph-mon[32761]: osd.3 v2:172.21.15.115:6813/3162056893 boot 2024-09-10T18:19:57.196 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:57 smithi150 ceph-mon[32761]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T18:19:57.197 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:57 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:19:57.219 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_4 2024-09-10T18:19:57.448 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:57 smithi115 ceph-mon[29603]: purged_snaps scrub starts 2024-09-10T18:19:57.448 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:57 smithi115 ceph-mon[29603]: purged_snaps scrub ok 2024-09-10T18:19:57.448 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:57 smithi115 ceph-mon[29603]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:19:57.448 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:57 smithi115 ceph-mon[29603]: osd.3 v2:172.21.15.115:6813/3162056893 boot 2024-09-10T18:19:57.448 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:57 smithi115 ceph-mon[29603]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T18:19:57.448 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:57 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:19:57.449 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:57 smithi115 ceph-mon[41073]: purged_snaps scrub starts 2024-09-10T18:19:57.449 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:57 smithi115 ceph-mon[41073]: purged_snaps scrub ok 2024-09-10T18:19:57.449 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:57 smithi115 ceph-mon[41073]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:19:57.449 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:57 smithi115 ceph-mon[41073]: osd.3 v2:172.21.15.115:6813/3162056893 boot 2024-09-10T18:19:57.449 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:57 smithi115 ceph-mon[41073]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T18:19:57.449 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:57 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:19:57.454 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:19:58.260 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:58 smithi150 ceph-mon[32761]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T18:19:58.260 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:58 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:58.261 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:58 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:58.261 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:58 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:19:58.261 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:58 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:19:58.261 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:58 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:19:58.261 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:58 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:19:58.479 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:58 smithi115 ceph-mon[41073]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T18:19:58.480 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:58 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:58.480 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:58 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:58.480 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:58 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:19:58.480 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:58 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:19:58.480 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:58 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:19:58.480 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:58 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:19:58.480 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:58 smithi115 ceph-mon[29603]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T18:19:58.480 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:58 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:58.480 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:58 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:58.481 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:58 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:19:58.481 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:58 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:19:58.481 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:58 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:19:58.481 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:58 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:19:59.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:59 smithi115 ceph-mon[29603]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:19:59.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:59 smithi115 ceph-mon[29603]: Detected new or changed devices on smithi115 2024-09-10T18:19:59.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:59 smithi115 ceph-mon[29603]: Adjusting osd_memory_target on smithi115 to 4016M 2024-09-10T18:19:59.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:59 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:59.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:59 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:59.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:59 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:19:59.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:59 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:59.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:59 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:19:59.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:59 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:59.488 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:59 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:19:59.488 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:19:59 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:59.488 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:59 smithi115 ceph-mon[41073]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:19:59.488 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:59 smithi115 ceph-mon[41073]: Detected new or changed devices on smithi115 2024-09-10T18:19:59.488 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:59 smithi115 ceph-mon[41073]: Adjusting osd_memory_target on smithi115 to 4016M 2024-09-10T18:19:59.488 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:59 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:59.488 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:59 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:59.488 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:59 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:19:59.488 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:59 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:59.488 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:59 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:19:59.488 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:59 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:59.489 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:59 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:19:59.489 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:19:59 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:59.552 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:59 smithi150 ceph-mon[32761]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:19:59.552 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:59 smithi150 ceph-mon[32761]: Detected new or changed devices on smithi115 2024-09-10T18:19:59.553 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:59 smithi150 ceph-mon[32761]: Adjusting osd_memory_target on smithi115 to 4016M 2024-09-10T18:19:59.553 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:59 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:59.553 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:59 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:59.553 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:59 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:19:59.553 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:59 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:19:59.553 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:59 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:19:59.553 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:59 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:59.553 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:59 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:19:59.553 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:19:59 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:00.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:00 smithi150 ceph-mon[32761]: from='client.24184 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:20:00.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:00 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:20:00.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:00 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:20:00.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:00 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:20:00.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:00 smithi150 ceph-mon[32761]: overall HEALTH_OK 2024-09-10T18:20:00.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:00 smithi115 ceph-mon[29603]: from='client.24184 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:20:00.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:00 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:20:00.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:00 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:20:00.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:00 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:20:00.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:00 smithi115 ceph-mon[29603]: overall HEALTH_OK 2024-09-10T18:20:00.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:00 smithi115 ceph-mon[41073]: from='client.24184 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:20:00.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:00 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:20:00.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:00 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:20:00.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:00 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:20:00.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:00 smithi115 ceph-mon[41073]: overall HEALTH_OK 2024-09-10T18:20:01.298 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:01 smithi150 ceph-mon[32761]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:20:01.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:01 smithi115 ceph-mon[29603]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:20:01.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:01 smithi115 ceph-mon[41073]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:20:02.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:02 smithi150 ceph-mon[32761]: from='client.? 172.21.15.150:0/2505669125' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa14f44d-1b45-490d-a686-77da88336564"}]: dispatch 2024-09-10T18:20:02.282 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:02 smithi150 ceph-mon[32761]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa14f44d-1b45-490d-a686-77da88336564"}]: dispatch 2024-09-10T18:20:02.282 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:02 smithi150 ceph-mon[32761]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa14f44d-1b45-490d-a686-77da88336564"}]': finished 2024-09-10T18:20:02.282 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:02 smithi150 ceph-mon[32761]: osdmap e27: 5 total, 4 up, 5 in 2024-09-10T18:20:02.282 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:02 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:20:02.282 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:02 smithi150 ceph-mon[32761]: from='client.? 172.21.15.150:0/4257115427' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:20:02.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:02 smithi115 ceph-mon[41073]: from='client.? 172.21.15.150:0/2505669125' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa14f44d-1b45-490d-a686-77da88336564"}]: dispatch 2024-09-10T18:20:02.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:02 smithi115 ceph-mon[41073]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa14f44d-1b45-490d-a686-77da88336564"}]: dispatch 2024-09-10T18:20:02.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:02 smithi115 ceph-mon[41073]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa14f44d-1b45-490d-a686-77da88336564"}]': finished 2024-09-10T18:20:02.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:02 smithi115 ceph-mon[41073]: osdmap e27: 5 total, 4 up, 5 in 2024-09-10T18:20:02.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:02 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:20:02.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:02 smithi115 ceph-mon[41073]: from='client.? 172.21.15.150:0/4257115427' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:20:02.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:02 smithi115 ceph-mon[29603]: from='client.? 172.21.15.150:0/2505669125' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa14f44d-1b45-490d-a686-77da88336564"}]: dispatch 2024-09-10T18:20:02.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:02 smithi115 ceph-mon[29603]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa14f44d-1b45-490d-a686-77da88336564"}]: dispatch 2024-09-10T18:20:02.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:02 smithi115 ceph-mon[29603]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa14f44d-1b45-490d-a686-77da88336564"}]': finished 2024-09-10T18:20:02.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:02 smithi115 ceph-mon[29603]: osdmap e27: 5 total, 4 up, 5 in 2024-09-10T18:20:02.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:02 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:20:02.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:02 smithi115 ceph-mon[29603]: from='client.? 172.21.15.150:0/4257115427' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:20:03.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:03 smithi115 ceph-mon[29603]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:20:03.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:03 smithi115 ceph-mon[41073]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:20:03.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:03 smithi150 ceph-mon[32761]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:20:05.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:05 smithi150 ceph-mon[32761]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:20:05.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:05 smithi115 ceph-mon[29603]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:20:05.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:05 smithi115 ceph-mon[41073]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:20:07.417 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:07 smithi150 ceph-mon[32761]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:20:07.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:07 smithi115 ceph-mon[29603]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:20:07.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:07 smithi115 ceph-mon[41073]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:20:08.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:08 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T18:20:08.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:08 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:20:08.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:08 smithi150 ceph-mon[32761]: Deploying daemon osd.4 on smithi150 2024-09-10T18:20:08.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:08 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T18:20:08.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:08 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:20:08.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:08 smithi115 ceph-mon[29603]: Deploying daemon osd.4 on smithi150 2024-09-10T18:20:08.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:08 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T18:20:08.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:08 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:20:08.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:08 smithi115 ceph-mon[41073]: Deploying daemon osd.4 on smithi150 2024-09-10T18:20:09.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:09 smithi150 ceph-mon[32761]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:20:09.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:09 smithi115 ceph-mon[41073]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:20:09.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:09 smithi115 ceph-mon[29603]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:20:11.619 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:11 smithi150 ceph-mon[32761]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:20:11.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:11 smithi115 ceph-mon[29603]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:20:11.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:11 smithi115 ceph-mon[41073]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:20:12.629 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:12 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:20:12.629 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:12 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:12.629 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:12 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:12.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:12 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:20:12.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:12 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:12.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:12 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:12.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:12 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:20:12.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:12 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:12.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:12 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:13.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:13 smithi115 ceph-mon[41073]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:20:13.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:13 smithi115 ceph-mon[29603]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:20:13.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:13 smithi150 ceph-mon[32761]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:20:15.488 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:15 smithi150 ceph-mon[32761]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:20:15.488 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:15 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:15.488 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:15 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:15.488 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:15 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:20:15.488 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:15 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:20:15.489 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:15 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:15.489 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:15 smithi150 ceph-mon[32761]: from='osd.4 v2:172.21.15.150:6800/1929277477' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:20:15.489 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:15 smithi150 ceph-mon[32761]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:20:15.528 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 4 on host 'smithi150' 2024-09-10T18:20:15.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:15 smithi115 ceph-mon[29603]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:20:15.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:15 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:15.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:15 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:15.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:15 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:20:15.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:15 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:20:15.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:15 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:15.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:15 smithi115 ceph-mon[29603]: from='osd.4 v2:172.21.15.150:6800/1929277477' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:20:15.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:15 smithi115 ceph-mon[29603]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:20:15.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:15 smithi115 ceph-mon[41073]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:20:15.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:15 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:15.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:15 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:15.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:15 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:20:15.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:15 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:20:15.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:15 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:15.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:15 smithi115 ceph-mon[41073]: from='osd.4 v2:172.21.15.150:6800/1929277477' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:20:15.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:15 smithi115 ceph-mon[41073]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:20:16.073 DEBUG:teuthology.orchestra.run.smithi150:osd.4> sudo journalctl -f -n 0 -u ceph-a600e180-6fa0-11ef-bcea-c7b262605968@osd.4.service 2024-09-10T18:20:16.076 INFO:tasks.cephadm:Deploying osd.5 on smithi150 with /dev/vg_nvme/lv_3... 2024-09-10T18:20:16.076 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-10T18:20:16.636 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:16 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:20:16.636 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:16 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:16.636 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:16 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:16.637 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:16 smithi115 ceph-mon[41073]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T18:20:16.637 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:16 smithi115 ceph-mon[41073]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T18:20:16.637 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:16 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:20:16.637 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:16 smithi115 ceph-mon[41073]: from='osd.4 v2:172.21.15.150:6800/1929277477' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T18:20:16.637 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:16 smithi115 ceph-mon[41073]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T18:20:16.637 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:16 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:20:16.637 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:16 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:16.637 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:16 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:16.637 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:16 smithi115 ceph-mon[29603]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T18:20:16.637 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:16 smithi115 ceph-mon[29603]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T18:20:16.638 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:16 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:20:16.638 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:16 smithi115 ceph-mon[29603]: from='osd.4 v2:172.21.15.150:6800/1929277477' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T18:20:16.638 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:16 smithi115 ceph-mon[29603]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T18:20:16.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:16 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:20:16.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:16 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:16.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:16 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:16.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:16 smithi150 ceph-mon[32761]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T18:20:16.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:16 smithi150 ceph-mon[32761]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T18:20:16.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:16 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:20:16.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:16 smithi150 ceph-mon[32761]: from='osd.4 v2:172.21.15.150:6800/1929277477' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T18:20:16.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:16 smithi150 ceph-mon[32761]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T18:20:16.833 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:20:17.098 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 18:20:16 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-4[38186]: 2024-09-10T18:20:16.838+0000 7f9fd7c71640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T18:20:17.098 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 18:20:16 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-4[38186]: 2024-09-10T18:20:16.842+0000 7f9fd0aa4640 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:20:17.628 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:17 smithi150 ceph-mon[32761]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:20:17.628 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:17 smithi150 ceph-mon[32761]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-09-10T18:20:17.628 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:17 smithi150 ceph-mon[32761]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T18:20:17.628 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:17 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:20:17.629 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:17 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:20:17.629 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:17 smithi150 ceph-mon[32761]: osd.4 v2:172.21.15.150:6800/1929277477 boot 2024-09-10T18:20:17.629 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:17 smithi150 ceph-mon[32761]: osdmap e30: 5 total, 5 up, 5 in 2024-09-10T18:20:17.629 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:17 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:20:17.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:17 smithi115 ceph-mon[29603]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:20:17.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:17 smithi115 ceph-mon[29603]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-09-10T18:20:17.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:17 smithi115 ceph-mon[29603]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T18:20:17.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:17 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:20:17.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:17 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:20:17.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:17 smithi115 ceph-mon[29603]: osd.4 v2:172.21.15.150:6800/1929277477 boot 2024-09-10T18:20:17.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:17 smithi115 ceph-mon[29603]: osdmap e30: 5 total, 5 up, 5 in 2024-09-10T18:20:17.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:17 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:20:17.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:17 smithi115 ceph-mon[41073]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:20:17.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:17 smithi115 ceph-mon[41073]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-09-10T18:20:17.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:17 smithi115 ceph-mon[41073]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T18:20:17.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:17 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:20:17.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:17 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:20:17.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:17 smithi115 ceph-mon[41073]: osd.4 v2:172.21.15.150:6800/1929277477 boot 2024-09-10T18:20:17.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:17 smithi115 ceph-mon[41073]: osdmap e30: 5 total, 5 up, 5 in 2024-09-10T18:20:17.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:17 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:20:19.196 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:18 smithi150 ceph-mon[32761]: purged_snaps scrub starts 2024-09-10T18:20:19.196 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:18 smithi150 ceph-mon[32761]: purged_snaps scrub ok 2024-09-10T18:20:19.196 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:18 smithi150 ceph-mon[32761]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T18:20:19.196 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:18 smithi150 ceph-mon[32761]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:20:19.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:18 smithi115 ceph-mon[29603]: purged_snaps scrub starts 2024-09-10T18:20:19.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:18 smithi115 ceph-mon[29603]: purged_snaps scrub ok 2024-09-10T18:20:19.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:18 smithi115 ceph-mon[29603]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T18:20:19.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:18 smithi115 ceph-mon[29603]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:20:19.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:18 smithi115 ceph-mon[41073]: purged_snaps scrub starts 2024-09-10T18:20:19.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:18 smithi115 ceph-mon[41073]: purged_snaps scrub ok 2024-09-10T18:20:19.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:18 smithi115 ceph-mon[41073]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T18:20:19.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:18 smithi115 ceph-mon[41073]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:20:20.212 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-10T18:20:20.213 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:19 smithi150 ceph-mon[32761]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T18:20:20.213 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:19 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:20.213 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:19 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:20.213 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:19 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:20:20.213 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:19 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:20.213 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:19 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:20:20.213 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:19 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:20:20.213 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:19 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:20.308 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-10T18:20:20.321 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-09-10T18:20:20.321 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-09-10T18:20:20.321 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114713 s, 914 MB/s 2024-09-10T18:20:20.321 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-09-10T18:20:20.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:19 smithi115 ceph-mon[29603]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T18:20:20.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:19 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:20.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:19 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:20.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:19 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:20:20.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:19 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:20.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:19 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:20:20.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:19 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:20:20.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:19 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:20.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:19 smithi115 ceph-mon[41073]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T18:20:20.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:19 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:20.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:19 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:20.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:19 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:20:20.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:19 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:20.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:19 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:20:20.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:19 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:20:20.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:19 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:20.943 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_3 2024-09-10T18:20:21.182 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:20:21.303 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:20 smithi150 ceph-mon[32761]: Detected new or changed devices on smithi150 2024-09-10T18:20:21.303 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:20 smithi150 ceph-mon[32761]: Adjusting osd_memory_target on smithi150 to 17088M 2024-09-10T18:20:21.303 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:20 smithi150 ceph-mon[32761]: pgmap v111: 1 pgs: 1 remapped+peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:20:21.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:20 smithi115 ceph-mon[29603]: Detected new or changed devices on smithi150 2024-09-10T18:20:21.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:20 smithi115 ceph-mon[29603]: Adjusting osd_memory_target on smithi150 to 17088M 2024-09-10T18:20:21.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:20 smithi115 ceph-mon[29603]: pgmap v111: 1 pgs: 1 remapped+peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:20:21.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:20 smithi115 ceph-mon[41073]: Detected new or changed devices on smithi150 2024-09-10T18:20:21.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:20 smithi115 ceph-mon[41073]: Adjusting osd_memory_target on smithi150 to 17088M 2024-09-10T18:20:21.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:20 smithi115 ceph-mon[41073]: pgmap v111: 1 pgs: 1 remapped+peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:20:23.401 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:23 smithi150 ceph-mon[32761]: pgmap v112: 1 pgs: 1 remapped+peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:20:23.401 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:23 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:20:23.401 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:23 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:20:23.401 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:23 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:20:23.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:23 smithi115 ceph-mon[29603]: pgmap v112: 1 pgs: 1 remapped+peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:20:23.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:23 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:20:23.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:23 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:20:23.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:23 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:20:23.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:23 smithi115 ceph-mon[41073]: pgmap v112: 1 pgs: 1 remapped+peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:20:23.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:23 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:20:23.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:23 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:20:23.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:23 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:20:24.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:24 smithi150 ceph-mon[32761]: from='client.24211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:20:24.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:24 smithi115 ceph-mon[29603]: from='client.24211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:20:24.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:24 smithi115 ceph-mon[41073]: from='client.24211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:20:25.323 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:25 smithi150 ceph-mon[32761]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-10T18:20:25.323 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:25 smithi150 ceph-mon[32761]: from='client.? 172.21.15.150:0/500921960' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40cabcd5-6874-45d2-bfaf-d737f834e7f4"}]: dispatch 2024-09-10T18:20:25.323 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:25 smithi150 ceph-mon[32761]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40cabcd5-6874-45d2-bfaf-d737f834e7f4"}]: dispatch 2024-09-10T18:20:25.324 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:25 smithi150 ceph-mon[32761]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40cabcd5-6874-45d2-bfaf-d737f834e7f4"}]': finished 2024-09-10T18:20:25.324 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:25 smithi150 ceph-mon[32761]: osdmap e33: 6 total, 5 up, 6 in 2024-09-10T18:20:25.324 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:25 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:20:25.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:25 smithi115 ceph-mon[29603]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-10T18:20:25.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:25 smithi115 ceph-mon[29603]: from='client.? 172.21.15.150:0/500921960' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40cabcd5-6874-45d2-bfaf-d737f834e7f4"}]: dispatch 2024-09-10T18:20:25.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:25 smithi115 ceph-mon[29603]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40cabcd5-6874-45d2-bfaf-d737f834e7f4"}]: dispatch 2024-09-10T18:20:25.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:25 smithi115 ceph-mon[29603]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40cabcd5-6874-45d2-bfaf-d737f834e7f4"}]': finished 2024-09-10T18:20:25.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:25 smithi115 ceph-mon[29603]: osdmap e33: 6 total, 5 up, 6 in 2024-09-10T18:20:25.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:25 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:20:25.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:25 smithi115 ceph-mon[41073]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-10T18:20:25.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:25 smithi115 ceph-mon[41073]: from='client.? 172.21.15.150:0/500921960' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40cabcd5-6874-45d2-bfaf-d737f834e7f4"}]: dispatch 2024-09-10T18:20:25.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:25 smithi115 ceph-mon[41073]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40cabcd5-6874-45d2-bfaf-d737f834e7f4"}]: dispatch 2024-09-10T18:20:25.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:25 smithi115 ceph-mon[41073]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40cabcd5-6874-45d2-bfaf-d737f834e7f4"}]': finished 2024-09-10T18:20:25.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:25 smithi115 ceph-mon[41073]: osdmap e33: 6 total, 5 up, 6 in 2024-09-10T18:20:25.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:25 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:20:26.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:26 smithi115 ceph-mon[29603]: from='client.? 172.21.15.150:0/1323402502' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:20:26.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:26 smithi115 ceph-mon[41073]: from='client.? 172.21.15.150:0/1323402502' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:20:26.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:26 smithi150 ceph-mon[32761]: from='client.? 172.21.15.150:0/1323402502' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:20:27.380 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:27 smithi115 ceph-mon[41073]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T18:20:27.381 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:27 smithi115 ceph-mon[29603]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T18:20:27.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:27 smithi150 ceph-mon[32761]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T18:20:29.159 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:29 smithi150 ceph-mon[32761]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-10T18:20:29.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:29 smithi115 ceph-mon[29603]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-10T18:20:29.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:29 smithi115 ceph-mon[41073]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-10T18:20:31.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:31 smithi150 ceph-mon[32761]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T18:20:31.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:31 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T18:20:31.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:31 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:20:31.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:31 smithi115 ceph-mon[29603]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T18:20:31.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:31 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T18:20:31.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:31 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:20:31.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:31 smithi115 ceph-mon[41073]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T18:20:31.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:31 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T18:20:31.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:31 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:20:32.440 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:32 smithi150 ceph-mon[32761]: Deploying daemon osd.5 on smithi150 2024-09-10T18:20:32.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:32 smithi115 ceph-mon[29603]: Deploying daemon osd.5 on smithi150 2024-09-10T18:20:32.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:32 smithi115 ceph-mon[41073]: Deploying daemon osd.5 on smithi150 2024-09-10T18:20:33.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:33 smithi150 ceph-mon[32761]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T18:20:33.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:33 smithi115 ceph-mon[41073]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T18:20:33.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:33 smithi115 ceph-mon[29603]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T18:20:35.446 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:35 smithi150 ceph-mon[32761]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:20:35.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:35 smithi115 ceph-mon[41073]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:20:35.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:35 smithi115 ceph-mon[29603]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:20:36.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:36 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:20:36.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:36 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:36.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:36 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:36.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:36 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:20:36.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:36 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:36.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:36 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:36.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:36 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:20:36.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:36 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:36.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:36 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:37.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:37 smithi150 ceph-mon[32761]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:20:37.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:37 smithi115 ceph-mon[41073]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:20:37.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:37 smithi115 ceph-mon[29603]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:20:39.390 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:39 smithi150 ceph-mon[32761]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:20:39.390 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:39 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:39.390 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:39 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:39.390 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:39 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:20:39.390 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:39 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:20:39.391 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:39 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:39.391 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:39 smithi150 ceph-mon[32761]: from='osd.5 v2:172.21.15.150:6804/3667323015' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:20:39.391 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:39 smithi150 ceph-mon[32761]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:20:39.427 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 5 on host 'smithi150' 2024-09-10T18:20:39.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:39 smithi115 ceph-mon[29603]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:20:39.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:39 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:39.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:39 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:39.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:39 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:20:39.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:39 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:20:39.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:39 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:39.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:39 smithi115 ceph-mon[29603]: from='osd.5 v2:172.21.15.150:6804/3667323015' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:20:39.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:39 smithi115 ceph-mon[29603]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:20:39.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:39 smithi115 ceph-mon[41073]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:20:39.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:39 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:39.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:39 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:39.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:39 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:20:39.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:39 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:20:39.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:39 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:39.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:39 smithi115 ceph-mon[41073]: from='osd.5 v2:172.21.15.150:6804/3667323015' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:20:39.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:39 smithi115 ceph-mon[41073]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:20:40.042 DEBUG:teuthology.orchestra.run.smithi150:osd.5> sudo journalctl -f -n 0 -u ceph-a600e180-6fa0-11ef-bcea-c7b262605968@osd.5.service 2024-09-10T18:20:40.044 INFO:tasks.cephadm:Deploying osd.6 on smithi150 with /dev/vg_nvme/lv_2... 2024-09-10T18:20:40.044 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-10T18:20:40.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:40 smithi150 ceph-mon[32761]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T18:20:40.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:40 smithi150 ceph-mon[32761]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T18:20:40.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:40 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:20:40.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:40 smithi150 ceph-mon[32761]: from='osd.5 v2:172.21.15.150:6804/3667323015' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T18:20:40.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:40 smithi150 ceph-mon[32761]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T18:20:40.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:40 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:20:40.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:40 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:40.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:40 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:40.556 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:20:40 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5[44072]: 2024-09-10T18:20:40.364+0000 7fbda2787640 -1 osd.5 0 waiting for initial osdmap 2024-09-10T18:20:40.556 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:20:40 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5[44072]: 2024-09-10T18:20:40.368+0000 7fbd9bdbb640 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:20:40.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:40 smithi115 ceph-mon[29603]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T18:20:40.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:40 smithi115 ceph-mon[29603]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T18:20:40.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:40 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:20:40.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:40 smithi115 ceph-mon[29603]: from='osd.5 v2:172.21.15.150:6804/3667323015' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T18:20:40.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:40 smithi115 ceph-mon[29603]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T18:20:40.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:40 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:20:40.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:40 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:40.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:40 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:40.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:40 smithi115 ceph-mon[41073]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T18:20:40.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:40 smithi115 ceph-mon[41073]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T18:20:40.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:40 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:20:40.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:40 smithi115 ceph-mon[41073]: from='osd.5 v2:172.21.15.150:6804/3667323015' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T18:20:40.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:40 smithi115 ceph-mon[41073]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T18:20:40.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:40 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:20:40.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:40 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:40.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:40 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:40.836 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:20:41.628 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:41 smithi150 ceph-mon[32761]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:20:41.628 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:41 smithi150 ceph-mon[32761]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-09-10T18:20:41.628 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:41 smithi150 ceph-mon[32761]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T18:20:41.628 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:41 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:20:41.628 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:41 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:20:41.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:41 smithi115 ceph-mon[29603]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:20:41.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:41 smithi115 ceph-mon[29603]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-09-10T18:20:41.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:41 smithi115 ceph-mon[29603]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T18:20:41.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:41 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:20:41.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:41 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:20:41.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:41 smithi115 ceph-mon[41073]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:20:41.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:41 smithi115 ceph-mon[41073]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-09-10T18:20:41.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:41 smithi115 ceph-mon[41073]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T18:20:41.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:41 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:20:41.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:41 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:20:42.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:42 smithi150 ceph-mon[32761]: purged_snaps scrub starts 2024-09-10T18:20:42.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:42 smithi150 ceph-mon[32761]: purged_snaps scrub ok 2024-09-10T18:20:42.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:42 smithi150 ceph-mon[32761]: osd.5 v2:172.21.15.150:6804/3667323015 boot 2024-09-10T18:20:42.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:42 smithi150 ceph-mon[32761]: osdmap e36: 6 total, 6 up, 6 in 2024-09-10T18:20:42.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:42 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:20:42.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:42 smithi150 ceph-mon[32761]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T18:20:42.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:42 smithi115 ceph-mon[29603]: purged_snaps scrub starts 2024-09-10T18:20:42.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:42 smithi115 ceph-mon[29603]: purged_snaps scrub ok 2024-09-10T18:20:42.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:42 smithi115 ceph-mon[29603]: osd.5 v2:172.21.15.150:6804/3667323015 boot 2024-09-10T18:20:42.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:42 smithi115 ceph-mon[29603]: osdmap e36: 6 total, 6 up, 6 in 2024-09-10T18:20:42.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:42 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:20:42.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:42 smithi115 ceph-mon[29603]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T18:20:42.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:42 smithi115 ceph-mon[41073]: purged_snaps scrub starts 2024-09-10T18:20:42.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:42 smithi115 ceph-mon[41073]: purged_snaps scrub ok 2024-09-10T18:20:42.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:42 smithi115 ceph-mon[41073]: osd.5 v2:172.21.15.150:6804/3667323015 boot 2024-09-10T18:20:42.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:42 smithi115 ceph-mon[41073]: osdmap e36: 6 total, 6 up, 6 in 2024-09-10T18:20:42.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:42 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:20:42.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:42 smithi115 ceph-mon[41073]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T18:20:43.619 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-10T18:20:43.619 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:43 smithi150 ceph-mon[32761]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:20:43.620 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:43 smithi150 ceph-mon[32761]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T18:20:43.697 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-10T18:20:43.712 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-09-10T18:20:43.712 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-09-10T18:20:43.712 INFO:teuthology.orchestra.run.smithi150.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0128319 s, 817 MB/s 2024-09-10T18:20:43.712 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-09-10T18:20:43.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:43 smithi115 ceph-mon[41073]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:20:43.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:43 smithi115 ceph-mon[41073]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T18:20:43.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:43 smithi115 ceph-mon[29603]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:20:43.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:43 smithi115 ceph-mon[29603]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T18:20:44.579 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_2 2024-09-10T18:20:44.818 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:20:45.189 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:44 smithi150 ceph-mon[32761]: Detected new or changed devices on smithi150 2024-09-10T18:20:45.189 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:44 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:45.189 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:44 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:45.189 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:44 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:20:45.189 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:44 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:20:45.189 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:44 smithi150 ceph-mon[32761]: Adjusting osd_memory_target on smithi150 to 8544M 2024-09-10T18:20:45.189 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:44 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:45.189 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:44 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:20:45.189 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:44 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:20:45.189 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:44 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:45.189 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:44 smithi150 ceph-mon[32761]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:20:45.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:44 smithi115 ceph-mon[29603]: Detected new or changed devices on smithi150 2024-09-10T18:20:45.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:44 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:45.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:44 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:45.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:44 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:20:45.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:44 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:20:45.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:44 smithi115 ceph-mon[29603]: Adjusting osd_memory_target on smithi150 to 8544M 2024-09-10T18:20:45.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:44 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:45.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:44 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:20:45.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:44 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:20:45.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:44 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:45.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:44 smithi115 ceph-mon[29603]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:20:45.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:44 smithi115 ceph-mon[41073]: Detected new or changed devices on smithi150 2024-09-10T18:20:45.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:44 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:45.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:44 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:45.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:44 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:20:45.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:44 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:20:45.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:44 smithi115 ceph-mon[41073]: Adjusting osd_memory_target on smithi150 to 8544M 2024-09-10T18:20:45.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:44 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:45.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:44 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:20:45.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:44 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:20:45.237 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:44 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:20:45.237 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:44 smithi115 ceph-mon[41073]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:20:47.206 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:47 smithi150 ceph-mon[32761]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:20:47.206 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:47 smithi150 ceph-mon[32761]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:20:47.206 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:47 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:20:47.207 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:47 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:20:47.207 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:47 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:20:47.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:47 smithi115 ceph-mon[29603]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:20:47.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:47 smithi115 ceph-mon[29603]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:20:47.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:47 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:20:47.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:47 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:20:47.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:47 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:20:47.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:47 smithi115 ceph-mon[41073]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:20:47.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:47 smithi115 ceph-mon[41073]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:20:47.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:47 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:20:47.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:47 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:20:47.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:47 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:20:49.418 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:49 smithi150 ceph-mon[32761]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:20:49.418 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:49 smithi150 ceph-mon[32761]: from='client.? 172.21.15.150:0/2136435427' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d871feca-36dd-4180-becc-e3e0dd9b4319"}]: dispatch 2024-09-10T18:20:49.418 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:49 smithi150 ceph-mon[32761]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d871feca-36dd-4180-becc-e3e0dd9b4319"}]: dispatch 2024-09-10T18:20:49.418 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:49 smithi150 ceph-mon[32761]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d871feca-36dd-4180-becc-e3e0dd9b4319"}]': finished 2024-09-10T18:20:49.418 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:49 smithi150 ceph-mon[32761]: osdmap e39: 7 total, 6 up, 7 in 2024-09-10T18:20:49.418 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:49 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:20:49.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:49 smithi115 ceph-mon[29603]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:20:49.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:49 smithi115 ceph-mon[29603]: from='client.? 172.21.15.150:0/2136435427' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d871feca-36dd-4180-becc-e3e0dd9b4319"}]: dispatch 2024-09-10T18:20:49.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:49 smithi115 ceph-mon[29603]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d871feca-36dd-4180-becc-e3e0dd9b4319"}]: dispatch 2024-09-10T18:20:49.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:49 smithi115 ceph-mon[29603]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d871feca-36dd-4180-becc-e3e0dd9b4319"}]': finished 2024-09-10T18:20:49.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:49 smithi115 ceph-mon[29603]: osdmap e39: 7 total, 6 up, 7 in 2024-09-10T18:20:49.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:49 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:20:49.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:49 smithi115 ceph-mon[41073]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:20:49.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:49 smithi115 ceph-mon[41073]: from='client.? 172.21.15.150:0/2136435427' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d871feca-36dd-4180-becc-e3e0dd9b4319"}]: dispatch 2024-09-10T18:20:49.487 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:49 smithi115 ceph-mon[41073]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d871feca-36dd-4180-becc-e3e0dd9b4319"}]: dispatch 2024-09-10T18:20:49.487 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:49 smithi115 ceph-mon[41073]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d871feca-36dd-4180-becc-e3e0dd9b4319"}]': finished 2024-09-10T18:20:49.487 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:49 smithi115 ceph-mon[41073]: osdmap e39: 7 total, 6 up, 7 in 2024-09-10T18:20:49.487 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:49 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:20:50.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:50 smithi115 ceph-mon[29603]: from='client.? 172.21.15.150:0/1703341167' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:20:50.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:50 smithi115 ceph-mon[41073]: from='client.? 172.21.15.150:0/1703341167' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:20:50.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:50 smithi150 ceph-mon[32761]: from='client.? 172.21.15.150:0/1703341167' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:20:51.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:51 smithi115 ceph-mon[41073]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:20:51.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:51 smithi115 ceph-mon[29603]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:20:51.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:51 smithi150 ceph-mon[32761]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:20:53.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:53 smithi150 ceph-mon[32761]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:20:53.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:53 smithi115 ceph-mon[29603]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:20:53.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:53 smithi115 ceph-mon[41073]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:20:55.375 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:55 smithi150 ceph-mon[32761]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:20:55.376 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:55 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T18:20:55.376 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:55 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:20:55.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:55 smithi115 ceph-mon[29603]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:20:55.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:55 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T18:20:55.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:55 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:20:55.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:55 smithi115 ceph-mon[41073]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:20:55.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:55 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T18:20:55.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:55 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:20:56.103 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:56 smithi150 ceph-mon[32761]: Deploying daemon osd.6 on smithi150 2024-09-10T18:20:56.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:56 smithi115 ceph-mon[29603]: Deploying daemon osd.6 on smithi150 2024-09-10T18:20:56.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:56 smithi115 ceph-mon[41073]: Deploying daemon osd.6 on smithi150 2024-09-10T18:20:57.273 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:57 smithi150 ceph-mon[32761]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:20:57.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:57 smithi115 ceph-mon[29603]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:20:57.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:57 smithi115 ceph-mon[41073]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:20:59.401 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:20:59 smithi150 ceph-mon[32761]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:20:59.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:20:59 smithi115 ceph-mon[29603]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:20:59.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:20:59 smithi115 ceph-mon[41073]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:21:00.298 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:00 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:21:00.298 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:00 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:00.298 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:00 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:00.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:00 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:21:00.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:00 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:00.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:00 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:00.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:00 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:21:00.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:00 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:00.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:00 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:01.277 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:01 smithi150 ceph-mon[32761]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:21:01.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:01 smithi115 ceph-mon[29603]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:21:01.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:01 smithi115 ceph-mon[41073]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:21:03.087 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 6 on host 'smithi150' 2024-09-10T18:21:03.674 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:03 smithi150 ceph-mon[32761]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:21:03.674 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:03 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:03.674 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:03 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:03.674 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:03 smithi150 ceph-mon[32761]: from='osd.6 v2:172.21.15.150:6808/3335870479' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:21:03.674 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:03 smithi150 ceph-mon[32761]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:21:03.675 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:03 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:03.675 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:03 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:03.697 DEBUG:teuthology.orchestra.run.smithi150:osd.6> sudo journalctl -f -n 0 -u ceph-a600e180-6fa0-11ef-bcea-c7b262605968@osd.6.service 2024-09-10T18:21:03.699 INFO:tasks.cephadm:Deploying osd.7 on smithi150 with /dev/vg_nvme/lv_1... 2024-09-10T18:21:03.700 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-10T18:21:03.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:03 smithi115 ceph-mon[41073]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:21:03.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:03 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:03.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:03 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:03.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:03 smithi115 ceph-mon[41073]: from='osd.6 v2:172.21.15.150:6808/3335870479' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:21:03.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:03 smithi115 ceph-mon[41073]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:21:03.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:03 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:03.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:03 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:03.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:03 smithi115 ceph-mon[29603]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:21:03.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:03 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:03.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:03 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:03.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:03 smithi115 ceph-mon[29603]: from='osd.6 v2:172.21.15.150:6808/3335870479' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:21:03.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:03 smithi115 ceph-mon[29603]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:21:03.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:03 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:03.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:03 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:03.994 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:21:04.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:04 smithi115 ceph-mon[41073]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T18:21:04.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:04 smithi115 ceph-mon[41073]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T18:21:04.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:04 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:21:04.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:04 smithi115 ceph-mon[41073]: from='osd.6 v2:172.21.15.150:6808/3335870479' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T18:21:04.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:04 smithi115 ceph-mon[41073]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T18:21:04.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:04 smithi115 ceph-mon[29603]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T18:21:04.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:04 smithi115 ceph-mon[29603]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T18:21:04.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:04 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:21:04.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:04 smithi115 ceph-mon[29603]: from='osd.6 v2:172.21.15.150:6808/3335870479' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T18:21:04.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:04 smithi115 ceph-mon[29603]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T18:21:04.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:04 smithi150 ceph-mon[32761]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T18:21:04.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:04 smithi150 ceph-mon[32761]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T18:21:04.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:04 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:21:04.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:04 smithi150 ceph-mon[32761]: from='osd.6 v2:172.21.15.150:6808/3335870479' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T18:21:04.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:04 smithi150 ceph-mon[32761]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T18:21:04.805 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:21:04 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:21:04.396+0000 7f017d1af640 -1 osd.6 0 waiting for initial osdmap 2024-09-10T18:21:04.806 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:21:04 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:21:04.401+0000 7f0177fe6640 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:21:05.602 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:05 smithi150 ceph-mon[32761]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:21:05.603 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:05 smithi150 ceph-mon[32761]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-09-10T18:21:05.603 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:05 smithi150 ceph-mon[32761]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T18:21:05.603 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:05 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:21:05.603 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:05 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:21:05.603 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:05 smithi150 ceph-mon[32761]: osd.6 v2:172.21.15.150:6808/3335870479 boot 2024-09-10T18:21:05.603 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:05 smithi150 ceph-mon[32761]: osdmap e42: 7 total, 7 up, 7 in 2024-09-10T18:21:05.603 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:05 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:21:05.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:05 smithi115 ceph-mon[41073]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:21:05.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:05 smithi115 ceph-mon[41073]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-09-10T18:21:05.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:05 smithi115 ceph-mon[41073]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T18:21:05.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:05 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:21:05.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:05 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:21:05.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:05 smithi115 ceph-mon[41073]: osd.6 v2:172.21.15.150:6808/3335870479 boot 2024-09-10T18:21:05.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:05 smithi115 ceph-mon[41073]: osdmap e42: 7 total, 7 up, 7 in 2024-09-10T18:21:05.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:05 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:21:05.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:05 smithi115 ceph-mon[29603]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:21:05.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:05 smithi115 ceph-mon[29603]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-09-10T18:21:05.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:05 smithi115 ceph-mon[29603]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T18:21:05.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:05 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:21:05.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:05 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:21:05.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:05 smithi115 ceph-mon[29603]: osd.6 v2:172.21.15.150:6808/3335870479 boot 2024-09-10T18:21:05.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:05 smithi115 ceph-mon[29603]: osdmap e42: 7 total, 7 up, 7 in 2024-09-10T18:21:05.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:05 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:21:06.386 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-10T18:21:06.486 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:06 smithi150 ceph-mon[32761]: purged_snaps scrub starts 2024-09-10T18:21:06.486 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:06 smithi150 ceph-mon[32761]: purged_snaps scrub ok 2024-09-10T18:21:06.486 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:06 smithi150 ceph-mon[32761]: osdmap e43: 7 total, 7 up, 7 in 2024-09-10T18:21:06.486 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-10T18:21:06.500 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-09-10T18:21:06.500 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-09-10T18:21:06.500 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125626 s, 835 MB/s 2024-09-10T18:21:06.500 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-09-10T18:21:06.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:06 smithi115 ceph-mon[41073]: purged_snaps scrub starts 2024-09-10T18:21:06.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:06 smithi115 ceph-mon[41073]: purged_snaps scrub ok 2024-09-10T18:21:06.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:06 smithi115 ceph-mon[41073]: osdmap e43: 7 total, 7 up, 7 in 2024-09-10T18:21:06.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:06 smithi115 ceph-mon[29603]: purged_snaps scrub starts 2024-09-10T18:21:06.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:06 smithi115 ceph-mon[29603]: purged_snaps scrub ok 2024-09-10T18:21:06.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:06 smithi115 ceph-mon[29603]: osdmap e43: 7 total, 7 up, 7 in 2024-09-10T18:21:07.540 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:07 smithi150 ceph-mon[32761]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:07.567 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_1 2024-09-10T18:21:07.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:07 smithi115 ceph-mon[29603]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:07.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:07 smithi115 ceph-mon[41073]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:07.817 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:21:08.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:08 smithi115 ceph-mon[29603]: osdmap e44: 7 total, 7 up, 7 in 2024-09-10T18:21:08.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:08 smithi115 ceph-mon[41073]: osdmap e44: 7 total, 7 up, 7 in 2024-09-10T18:21:08.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:08 smithi150 ceph-mon[32761]: osdmap e44: 7 total, 7 up, 7 in 2024-09-10T18:21:09.729 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:09 smithi150 ceph-mon[32761]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:09.729 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:09 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:09.729 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:09 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:09.729 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:09 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:21:09.729 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:09 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:21:09.729 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:09 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:21:09.730 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:09 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:09.730 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:09 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:21:09.730 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:09 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:21:09.730 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:09 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:09.730 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:09 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:21:09.730 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:09 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:21:09.730 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:09 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:21:09.730 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:09 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:09.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:09 smithi115 ceph-mon[29603]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:09.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:09 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:09.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:09 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:09.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:09 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:21:09.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:09 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:21:09.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:09 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:21:09.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:09 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:09.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:09 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:21:09.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:09 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:21:09.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:09 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:09.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:09 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:21:09.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:09 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:21:09.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:09 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:21:09.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:09 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:09.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:09 smithi115 ceph-mon[41073]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:09.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:09 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:09.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:09 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:09.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:09 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:21:09.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:09 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:21:09.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:09 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:21:09.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:09 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:09.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:09 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:21:09.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:09 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:21:09.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:09 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:09.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:09 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:21:09.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:09 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:21:09.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:09 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:21:09.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:09 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:10.715 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:10 smithi150 ceph-mon[32761]: Detected new or changed devices on smithi150 2024-09-10T18:21:10.715 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:10 smithi150 ceph-mon[32761]: Adjusting osd_memory_target on smithi150 to 5696M 2024-09-10T18:21:10.715 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:10 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:21:10.715 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:10 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:21:10.715 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:10 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:21:10.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:10 smithi115 ceph-mon[29603]: Detected new or changed devices on smithi150 2024-09-10T18:21:10.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:10 smithi115 ceph-mon[29603]: Adjusting osd_memory_target on smithi150 to 5696M 2024-09-10T18:21:10.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:10 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:21:10.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:10 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:21:10.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:10 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:21:10.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:10 smithi115 ceph-mon[41073]: Detected new or changed devices on smithi150 2024-09-10T18:21:10.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:10 smithi115 ceph-mon[41073]: Adjusting osd_memory_target on smithi150 to 5696M 2024-09-10T18:21:10.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:10 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:21:10.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:10 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:21:10.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:10 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:21:11.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:11 smithi150 ceph-mon[32761]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:11.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:11 smithi150 ceph-mon[32761]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:21:11.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:11 smithi115 ceph-mon[29603]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:11.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:11 smithi115 ceph-mon[29603]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:21:11.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:11 smithi115 ceph-mon[41073]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:11.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:11 smithi115 ceph-mon[41073]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:21:13.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:13 smithi115 ceph-mon[41073]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:13.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:13 smithi115 ceph-mon[41073]: from='client.? 172.21.15.150:0/3053474814' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0aaa1287-7337-49a5-a832-709aa3980834"}]: dispatch 2024-09-10T18:21:13.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:13 smithi115 ceph-mon[41073]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0aaa1287-7337-49a5-a832-709aa3980834"}]: dispatch 2024-09-10T18:21:13.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:13 smithi115 ceph-mon[41073]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0aaa1287-7337-49a5-a832-709aa3980834"}]': finished 2024-09-10T18:21:13.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:13 smithi115 ceph-mon[41073]: osdmap e45: 8 total, 7 up, 8 in 2024-09-10T18:21:13.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:13 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:21:13.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:13 smithi115 ceph-mon[41073]: from='client.? 172.21.15.150:0/278655612' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:21:13.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:13 smithi115 ceph-mon[29603]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:13.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:13 smithi115 ceph-mon[29603]: from='client.? 172.21.15.150:0/3053474814' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0aaa1287-7337-49a5-a832-709aa3980834"}]: dispatch 2024-09-10T18:21:13.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:13 smithi115 ceph-mon[29603]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0aaa1287-7337-49a5-a832-709aa3980834"}]: dispatch 2024-09-10T18:21:13.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:13 smithi115 ceph-mon[29603]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0aaa1287-7337-49a5-a832-709aa3980834"}]': finished 2024-09-10T18:21:13.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:13 smithi115 ceph-mon[29603]: osdmap e45: 8 total, 7 up, 8 in 2024-09-10T18:21:13.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:13 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:21:13.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:13 smithi115 ceph-mon[29603]: from='client.? 172.21.15.150:0/278655612' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:21:13.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:13 smithi150 ceph-mon[32761]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:13.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:13 smithi150 ceph-mon[32761]: from='client.? 172.21.15.150:0/3053474814' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0aaa1287-7337-49a5-a832-709aa3980834"}]: dispatch 2024-09-10T18:21:13.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:13 smithi150 ceph-mon[32761]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0aaa1287-7337-49a5-a832-709aa3980834"}]: dispatch 2024-09-10T18:21:13.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:13 smithi150 ceph-mon[32761]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0aaa1287-7337-49a5-a832-709aa3980834"}]': finished 2024-09-10T18:21:13.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:13 smithi150 ceph-mon[32761]: osdmap e45: 8 total, 7 up, 8 in 2024-09-10T18:21:13.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:13 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:21:13.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:13 smithi150 ceph-mon[32761]: from='client.? 172.21.15.150:0/278655612' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:21:15.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:15 smithi115 ceph-mon[29603]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:15.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:15 smithi115 ceph-mon[41073]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:15.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:15 smithi150 ceph-mon[32761]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:17.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:17 smithi150 ceph-mon[32761]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:17.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:17 smithi115 ceph-mon[41073]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:17.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:17 smithi115 ceph-mon[29603]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:19.426 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:19 smithi150 ceph-mon[32761]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:19.426 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:19 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T18:21:19.426 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:19 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:21:19.426 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:19 smithi150 ceph-mon[32761]: Deploying daemon osd.7 on smithi150 2024-09-10T18:21:19.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:19 smithi115 ceph-mon[29603]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:19.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:19 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T18:21:19.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:19 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:21:19.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:19 smithi115 ceph-mon[29603]: Deploying daemon osd.7 on smithi150 2024-09-10T18:21:19.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:19 smithi115 ceph-mon[41073]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:19.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:19 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T18:21:19.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:19 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:21:19.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:19 smithi115 ceph-mon[41073]: Deploying daemon osd.7 on smithi150 2024-09-10T18:21:21.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:21 smithi115 ceph-mon[29603]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:21.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:21 smithi115 ceph-mon[41073]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:21.738 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:21 smithi150 ceph-mon[32761]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:23.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:23 smithi150 ceph-mon[32761]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:23.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:23 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:21:23.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:23 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:23.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:23 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:23.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:23 smithi115 ceph-mon[41073]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:23.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:23 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:21:23.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:23 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:23.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:23 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:23.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:23 smithi115 ceph-mon[29603]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:23.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:23 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:21:23.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:23 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:23.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:23 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:25.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:25 smithi150 ceph-mon[32761]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:25.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:25 smithi115 ceph-mon[29603]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:25.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:25 smithi115 ceph-mon[41073]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:26.656 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 7 on host 'smithi150' 2024-09-10T18:21:27.286 DEBUG:teuthology.orchestra.run.smithi150:osd.7> sudo journalctl -f -n 0 -u ceph-a600e180-6fa0-11ef-bcea-c7b262605968@osd.7.service 2024-09-10T18:21:27.289 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-09-10T18:21:27.289 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T18:21:27.547 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:21:27.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:27 smithi150 ceph-mon[32761]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:27.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:27 smithi150 ceph-mon[32761]: from='osd.7 v2:172.21.15.150:6812/672367088' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:21:27.557 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:27 smithi150 ceph-mon[32761]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:21:27.557 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:27 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:27.557 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:27 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:27.668 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:27 smithi115 ceph-mon[41073]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:27.668 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:27 smithi115 ceph-mon[41073]: from='osd.7 v2:172.21.15.150:6812/672367088' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:21:27.669 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:27 smithi115 ceph-mon[41073]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:21:27.669 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:27 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:27.669 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:27 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:27.669 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:27 smithi115 ceph-mon[29603]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:27.669 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:27 smithi115 ceph-mon[29603]: from='osd.7 v2:172.21.15.150:6812/672367088' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:21:27.669 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:27 smithi115 ceph-mon[29603]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:21:27.669 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:27 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:27.669 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:27 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:28.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:28 smithi115 ceph-mon[29603]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T18:21:28.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:28 smithi115 ceph-mon[29603]: osdmap e46: 8 total, 7 up, 8 in 2024-09-10T18:21:28.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:28 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:21:28.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:28 smithi115 ceph-mon[29603]: from='osd.7 v2:172.21.15.150:6812/672367088' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T18:21:28.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:28 smithi115 ceph-mon[29603]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T18:21:28.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:28 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:28.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:28 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:28.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:28 smithi115 ceph-mon[41073]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T18:21:28.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:28 smithi115 ceph-mon[41073]: osdmap e46: 8 total, 7 up, 8 in 2024-09-10T18:21:28.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:28 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:21:28.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:28 smithi115 ceph-mon[41073]: from='osd.7 v2:172.21.15.150:6812/672367088' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T18:21:28.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:28 smithi115 ceph-mon[41073]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T18:21:28.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:28 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:28.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:28 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:28.789 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:28 smithi150 ceph-mon[32761]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T18:21:28.790 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:28 smithi150 ceph-mon[32761]: osdmap e46: 8 total, 7 up, 8 in 2024-09-10T18:21:28.790 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:28 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:21:28.790 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:28 smithi150 ceph-mon[32761]: from='osd.7 v2:172.21.15.150:6812/672367088' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T18:21:28.790 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:28 smithi150 ceph-mon[32761]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T18:21:28.790 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:28 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:28.790 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:28 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:28.790 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:21:28 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:21:28.452+0000 7f7f03b5c640 -1 osd.7 0 waiting for initial osdmap 2024-09-10T18:21:28.790 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:21:28 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:21:28.458+0000 7f7efd190640 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:21:29.593 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:21:29.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:29 smithi115 ceph-mon[41073]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:29.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:29 smithi115 ceph-mon[41073]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-09-10T18:21:29.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:29 smithi115 ceph-mon[41073]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T18:21:29.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:29 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:21:29.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:29 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:21:29.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:29 smithi115 ceph-mon[29603]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:29.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:29 smithi115 ceph-mon[29603]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-09-10T18:21:29.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:29 smithi115 ceph-mon[29603]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T18:21:29.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:29 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:21:29.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:29 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:21:29.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:29 smithi150 ceph-mon[32761]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:29.808 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:29 smithi150 ceph-mon[32761]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-09-10T18:21:29.808 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:29 smithi150 ceph-mon[32761]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T18:21:29.809 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:29 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:21:29.809 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:29 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:21:30.389 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":48,"num_osds":8,"num_up_osds":8,"osd_up_since":1725992489,"num_in_osds":8,"osd_in_since":1725992472,"num_remapped_pgs":1} 2024-09-10T18:21:30.390 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T18:21:30.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:30 smithi150 ceph-mon[32761]: purged_snaps scrub starts 2024-09-10T18:21:30.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:30 smithi150 ceph-mon[32761]: purged_snaps scrub ok 2024-09-10T18:21:30.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:30 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:21:30.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:30 smithi150 ceph-mon[32761]: osd.7 v2:172.21.15.150:6812/672367088 boot 2024-09-10T18:21:30.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:30 smithi150 ceph-mon[32761]: osdmap e48: 8 total, 8 up, 8 in 2024-09-10T18:21:30.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:30 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:21:30.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:30 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1654110777' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T18:21:30.636 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:21:30.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:30 smithi115 ceph-mon[41073]: purged_snaps scrub starts 2024-09-10T18:21:30.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:30 smithi115 ceph-mon[41073]: purged_snaps scrub ok 2024-09-10T18:21:30.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:30 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:21:30.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:30 smithi115 ceph-mon[41073]: osd.7 v2:172.21.15.150:6812/672367088 boot 2024-09-10T18:21:30.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:30 smithi115 ceph-mon[41073]: osdmap e48: 8 total, 8 up, 8 in 2024-09-10T18:21:30.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:30 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:21:30.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:30 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1654110777' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T18:21:30.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:30 smithi115 ceph-mon[29603]: purged_snaps scrub starts 2024-09-10T18:21:30.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:30 smithi115 ceph-mon[29603]: purged_snaps scrub ok 2024-09-10T18:21:30.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:30 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:21:30.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:30 smithi115 ceph-mon[29603]: osd.7 v2:172.21.15.150:6812/672367088 boot 2024-09-10T18:21:30.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:30 smithi115 ceph-mon[29603]: osdmap e48: 8 total, 8 up, 8 in 2024-09-10T18:21:30.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:30 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:21:30.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:30 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1654110777' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T18:21:31.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:31 smithi115 ceph-mon[29603]: pgmap v162: 1 pgs: 1 peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:31.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:31 smithi115 ceph-mon[29603]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T18:21:31.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:31 smithi115 ceph-mon[29603]: osdmap e49: 8 total, 8 up, 8 in 2024-09-10T18:21:31.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:31 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:31.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:31 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:31.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:31 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:21:31.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:31 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:21:31.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:31 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:21:31.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:31 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:21:31.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:31 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:31.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:31 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:21:31.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:31 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:21:31.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:31 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:31.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:31 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:21:31.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:31 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:21:31.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:31 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:21:31.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:31 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:31.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:31 smithi115 ceph-mon[41073]: pgmap v162: 1 pgs: 1 peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:31.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:31 smithi115 ceph-mon[41073]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T18:21:31.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:31 smithi115 ceph-mon[41073]: osdmap e49: 8 total, 8 up, 8 in 2024-09-10T18:21:31.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:31 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:31.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:31 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:31.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:31 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:21:31.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:31 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:21:31.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:31 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:21:31.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:31 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:21:31.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:31 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:31.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:31 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:21:31.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:31 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:21:31.738 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:31 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:31.738 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:31 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:21:31.738 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:31 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:21:31.738 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:31 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:21:31.738 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:31 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:31.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:31 smithi150 ceph-mon[32761]: pgmap v162: 1 pgs: 1 peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:21:31.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:31 smithi150 ceph-mon[32761]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T18:21:31.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:31 smithi150 ceph-mon[32761]: osdmap e49: 8 total, 8 up, 8 in 2024-09-10T18:21:31.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:31 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:31.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:31 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:31.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:31 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:21:31.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:31 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:21:31.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:31 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:21:31.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:31 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:21:31.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:31 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:31.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:31 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:21:31.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:31 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:21:31.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:31 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:31.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:31 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:21:31.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:31 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:21:31.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:31 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:21:31.807 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:31 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:32.528 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:21:32.528 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":50,"fsid":"a600e180-6fa0-11ef-bcea-c7b262605968","created":"2024-09-10T18:16:27.008296+0000","modified":"2024-09-10T18:21:31.465809+0000","last_up_change":"2024-09-10T18:21:29.447953+0000","last_in_change":"2024-09-10T18:21:12.478592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T18:19:32.280261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"107d9598-28cd-4260-9f5d-a3b7886845c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6801","nonce":4123495067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":4123495067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":4123495067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6803","nonce":4123495067}]},"public_addr":"172.21.15.115:6801/4123495067","cluster_addr":"172.21.15.115:6802/4123495067","heartbeat_back_addr":"172.21.15.115:6804/4123495067","heartbeat_front_addr":"172.21.15.115:6803/4123495067","state":["exists","up"]},{"osd":1,"uuid":"c3cfd69f-7300-4b6e-a57b-33eefed950b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6805","nonce":2506968391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2506968391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2506968391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6807","nonce":2506968391}]},"public_addr":"172.21.15.115:6805/2506968391","cluster_addr":"172.21.15.115:6806/2506968391","heartbeat_back_addr":"172.21.15.115:6808/2506968391","heartbeat_front_addr":"172.21.15.115:6807/2506968391","state":["exists","up"]},{"osd":2,"uuid":"9abe6878-817d-4c81-b4b8-db61147c2f94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6809","nonce":343149052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":343149052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":343149052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6811","nonce":343149052}]},"public_addr":"172.21.15.115:6809/343149052","cluster_addr":"172.21.15.115:6810/343149052","heartbeat_back_addr":"172.21.15.115:6812/343149052","heartbeat_front_addr":"172.21.15.115:6811/343149052","state":["exists","up"]},{"osd":3,"uuid":"4fefb92a-fc8d-4a28-9c89-1b7e214c8e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6813","nonce":3162056893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":3162056893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3162056893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":3162056893}]},"public_addr":"172.21.15.115:6813/3162056893","cluster_addr":"172.21.15.115:6814/3162056893","heartbeat_back_addr":"172.21.15.115:6816/3162056893","heartbeat_front_addr":"172.21.15.115:6815/3162056893","state":["exists","up"]},{"osd":4,"uuid":"fa14f44d-1b45-490d-a686-77da88336564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1929277477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6801","nonce":1929277477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6803","nonce":1929277477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1929277477}]},"public_addr":"172.21.15.150:6800/1929277477","cluster_addr":"172.21.15.150:6801/1929277477","heartbeat_back_addr":"172.21.15.150:6803/1929277477","heartbeat_front_addr":"172.21.15.150:6802/1929277477","state":["exists","up"]},{"osd":5,"uuid":"40cabcd5-6874-45d2-bfaf-d737f834e7f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3667323015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6805","nonce":3667323015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6807","nonce":3667323015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3667323015}]},"public_addr":"172.21.15.150:6804/3667323015","cluster_addr":"172.21.15.150:6805/3667323015","heartbeat_back_addr":"172.21.15.150:6807/3667323015","heartbeat_front_addr":"172.21.15.150:6806/3667323015","state":["exists","up"]},{"osd":6,"uuid":"d871feca-36dd-4180-becc-e3e0dd9b4319","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3335870479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3335870479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6811","nonce":3335870479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3335870479}]},"public_addr":"172.21.15.150:6808/3335870479","cluster_addr":"172.21.15.150:6809/3335870479","heartbeat_back_addr":"172.21.15.150:6811/3335870479","heartbeat_front_addr":"172.21.15.150:6810/3335870479","state":["exists","up"]},{"osd":7,"uuid":"0aaa1287-7337-49a5-a832-709aa3980834","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":672367088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":672367088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6815","nonce":672367088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":672367088}]},"public_addr":"172.21.15.150:6812/672367088","cluster_addr":"172.21.15.150:6813/672367088","heartbeat_back_addr":"172.21.15.150:6815/672367088","heartbeat_front_addr":"172.21.15.150:6814/672367088","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:18:41.264707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:19:05.260563+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:19:29.810350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:19:54.613874+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:20:16.294916+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:20:40.099086+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:21:03.703231+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:21:27.515712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/2382563730":"2024-09-11T18:17:21.977918+0000","172.21.15.115:0/3905115041":"2024-09-11T18:17:21.977918+0000","172.21.15.115:6800/3682910233":"2024-09-11T18:17:21.977918+0000","172.21.15.115:6800/2658904986":"2024-09-11T18:16:59.107583+0000","172.21.15.115:0/553409925":"2024-09-11T18:17:21.977918+0000","172.21.15.115:0/1020299194":"2024-09-11T18:16:59.107583+0000","172.21.15.115:0/3979277447":"2024-09-11T18:16:59.107583+0000","172.21.15.115:0/936574235":"2024-09-11T18:16:59.107583+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T18:21:32.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:32 smithi115 ceph-mon[29603]: Detected new or changed devices on smithi150 2024-09-10T18:21:32.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:32 smithi115 ceph-mon[29603]: Adjusting osd_memory_target on smithi150 to 4272M 2024-09-10T18:21:32.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:32 smithi115 ceph-mon[29603]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T18:21:32.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:32 smithi115 ceph-mon[41073]: Detected new or changed devices on smithi150 2024-09-10T18:21:32.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:32 smithi115 ceph-mon[41073]: Adjusting osd_memory_target on smithi150 to 4272M 2024-09-10T18:21:32.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:32 smithi115 ceph-mon[41073]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T18:21:32.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:32 smithi150 ceph-mon[32761]: Detected new or changed devices on smithi150 2024-09-10T18:21:32.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:32 smithi150 ceph-mon[32761]: Adjusting osd_memory_target on smithi150 to 4272M 2024-09-10T18:21:32.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:32 smithi150 ceph-mon[32761]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T18:21:33.261 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T18:19:32.280261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-09-10T18:21:33.262 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-10T18:21:33.502 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:21:33.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:33 smithi150 ceph-mon[32761]: pgmap v165: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:21:33.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:33 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/3482527895' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:21:33.623 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:33 smithi115 ceph-mon[29603]: pgmap v165: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:21:33.623 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:33 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/3482527895' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:21:33.624 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:33 smithi115 ceph-mon[41073]: pgmap v165: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:21:33.624 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:33 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/3482527895' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:21:35.380 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2024-09-10T18:21:35.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:35 smithi115 ceph-mon[29603]: pgmap v166: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:21:35.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:35 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/4275955077' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T18:21:35.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:35 smithi115 ceph-mon[41073]: pgmap v166: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:21:35.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:35 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/4275955077' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T18:21:35.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:35 smithi150 ceph-mon[32761]: pgmap v166: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:21:35.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:35 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/4275955077' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T18:21:36.059 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi115 2024-09-10T18:21:36.059 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph orch apply rgw foo.a --placement '1;smithi115=foo.a' 2024-09-10T18:21:36.315 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:21:36.733 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:36 smithi150 ceph-mon[32761]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T18:21:36.733 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:36 smithi150 ceph-mon[32761]: Cluster is now healthy 2024-09-10T18:21:36.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:36 smithi115 ceph-mon[29603]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T18:21:36.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:36 smithi115 ceph-mon[29603]: Cluster is now healthy 2024-09-10T18:21:36.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:36 smithi115 ceph-mon[41073]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T18:21:36.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:36 smithi115 ceph-mon[41073]: Cluster is now healthy 2024-09-10T18:21:37.777 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:37 smithi150 ceph-mon[32761]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-10T18:21:37.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:37 smithi115 ceph-mon[29603]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-10T18:21:37.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:37 smithi115 ceph-mon[41073]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-10T18:21:38.162 INFO:teuthology.orchestra.run.smithi150.stdout:Scheduled rgw.foo.a update... 2024-09-10T18:21:38.903 DEBUG:teuthology.orchestra.run.smithi115:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-a600e180-6fa0-11ef-bcea-c7b262605968@rgw.foo.a.service 2024-09-10T18:21:38.906 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi150 2024-09-10T18:21:38.906 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph osd pool create datapool 3 3 replicated 2024-09-10T18:21:39.150 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:21:39.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:39 smithi115 ceph-mon[29603]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T18:21:39.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:39 smithi115 ceph-mon[29603]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi115=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:21:39.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:39 smithi115 ceph-mon[29603]: Saving service rgw.foo.a spec with placement smithi115=foo.a;count:1 2024-09-10T18:21:39.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:39 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:39.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:39 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:21:39.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:39 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:21:39.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:39 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:21:39.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:39 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:39.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:39 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-10T18:21:39.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:39 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-10T18:21:39.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:39 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:39.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:39 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:21:39.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:39 smithi115 ceph-mon[29603]: Deploying daemon rgw.foo.a on smithi115 2024-09-10T18:21:39.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:39 smithi115 ceph-mon[41073]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T18:21:39.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:39 smithi115 ceph-mon[41073]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi115=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:21:39.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:39 smithi115 ceph-mon[41073]: Saving service rgw.foo.a spec with placement smithi115=foo.a;count:1 2024-09-10T18:21:39.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:39 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:39.487 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:39 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:21:39.487 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:39 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:21:39.487 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:39 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:21:39.487 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:39 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:39.487 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:39 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-10T18:21:39.487 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:39 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-10T18:21:39.487 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:39 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:39.487 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:39 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:21:39.487 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:39 smithi115 ceph-mon[41073]: Deploying daemon rgw.foo.a on smithi115 2024-09-10T18:21:39.505 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:39 smithi150 ceph-mon[32761]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T18:21:39.505 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:39 smithi150 ceph-mon[32761]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi115=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:21:39.505 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:39 smithi150 ceph-mon[32761]: Saving service rgw.foo.a spec with placement smithi115=foo.a;count:1 2024-09-10T18:21:39.505 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:39 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:39.505 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:39 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:21:39.505 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:39 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:21:39.505 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:39 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:21:39.505 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:39 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:39.505 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:39 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-10T18:21:39.505 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:39 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-10T18:21:39.506 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:39 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:39.506 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:39 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:21:39.506 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:39 smithi150 ceph-mon[32761]: Deploying daemon rgw.foo.a on smithi115 2024-09-10T18:21:40.485 INFO:journalctl@ceph.rgw.foo.a.smithi115.stdout:Sep 10 18:21:40 smithi115 podman[69388]: 2024-09-10 18:21:40.397381824 +0000 UTC m=+0.140588178 container create 31854f3a8e9ff3fda99bceace9bf21c1b547badb37df39130908571a36c3ebd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-rgw-foo-a, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-10T18:21:40.985 INFO:journalctl@ceph.rgw.foo.a.smithi115.stdout:Sep 10 18:21:40 smithi115 podman[69388]: 2024-09-10 18:21:40.635400581 +0000 UTC m=+0.378606936 container init 31854f3a8e9ff3fda99bceace9bf21c1b547badb37df39130908571a36c3ebd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-rgw-foo-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2) 2024-09-10T18:21:40.985 INFO:journalctl@ceph.rgw.foo.a.smithi115.stdout:Sep 10 18:21:40 smithi115 podman[69388]: 2024-09-10 18:21:40.640180778 +0000 UTC m=+0.383387136 container start 31854f3a8e9ff3fda99bceace9bf21c1b547badb37df39130908571a36c3ebd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-rgw-foo-a, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True) 2024-09-10T18:21:40.985 INFO:journalctl@ceph.rgw.foo.a.smithi115.stdout:Sep 10 18:21:40 smithi115 bash[69388]: 31854f3a8e9ff3fda99bceace9bf21c1b547badb37df39130908571a36c3ebd5 2024-09-10T18:21:40.985 INFO:journalctl@ceph.rgw.foo.a.smithi115.stdout:Sep 10 18:21:40 smithi115 systemd[1]: Started Ceph rgw.foo.a for a600e180-6fa0-11ef-bcea-c7b262605968. 2024-09-10T18:21:41.173 INFO:teuthology.orchestra.run.smithi150.stderr:pool 'datapool' created 2024-09-10T18:21:41.440 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:41 smithi150 ceph-mon[32761]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-10T18:21:41.440 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:41 smithi150 ceph-mon[32761]: from='client.? 172.21.15.150:0/1599534037' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:21:41.440 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:41 smithi150 ceph-mon[32761]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:21:41.440 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:41 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:41.441 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:41 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:41.441 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:41 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:41.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:41 smithi115 ceph-mon[41073]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-10T18:21:41.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:41 smithi115 ceph-mon[41073]: from='client.? 172.21.15.150:0/1599534037' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:21:41.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:41 smithi115 ceph-mon[41073]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:21:41.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:41 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:41.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:41 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:41.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:41 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:41.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:41 smithi115 ceph-mon[29603]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-10T18:21:41.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:41 smithi115 ceph-mon[29603]: from='client.? 172.21.15.150:0/1599534037' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:21:41.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:41 smithi115 ceph-mon[29603]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:21:41.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:41 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:41.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:41 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:41.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:41 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:41.789 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- rbd pool init datapool 2024-09-10T18:21:42.030 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:21:42.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:42 smithi115 ceph-mon[41073]: Saving service rgw.foo.a spec with placement smithi115=foo.a;count:1 2024-09-10T18:21:42.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:42 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:42.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:42 smithi115 ceph-mon[41073]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-10T18:21:42.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:42 smithi115 ceph-mon[41073]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T18:21:42.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:42 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/819676780' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T18:21:42.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:42 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:42.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:42 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:21:42.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:42 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:42.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:42 smithi115 ceph-mon[29603]: Saving service rgw.foo.a spec with placement smithi115=foo.a;count:1 2024-09-10T18:21:42.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:42 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:42.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:42 smithi115 ceph-mon[29603]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-10T18:21:42.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:42 smithi115 ceph-mon[29603]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T18:21:42.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:42 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/819676780' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T18:21:42.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:42 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:42.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:42 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:21:42.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:42 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:42.487 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:42 smithi150 ceph-mon[32761]: Saving service rgw.foo.a spec with placement smithi115=foo.a;count:1 2024-09-10T18:21:42.487 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:42 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:42.487 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:42 smithi150 ceph-mon[32761]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-10T18:21:42.487 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:42 smithi150 ceph-mon[32761]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T18:21:42.487 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:42 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/819676780' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T18:21:42.487 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:42 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:42.487 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:42 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:21:42.487 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:42 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:43.405 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:43 smithi150 ceph-mon[32761]: pgmap v171: 36 pgs: 5 creating+peering, 30 unknown, 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T18:21:43.405 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:43 smithi150 ceph-mon[32761]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:21:43.405 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:43 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/819676780' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-10T18:21:43.405 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:43 smithi150 ceph-mon[32761]: osdmap e52: 8 total, 8 up, 8 in 2024-09-10T18:21:43.405 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:43 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:43.405 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:43 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:43.405 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:43 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:21:43.405 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:43 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:21:43.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:43 smithi115 ceph-mon[41073]: pgmap v171: 36 pgs: 5 creating+peering, 30 unknown, 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T18:21:43.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:43 smithi115 ceph-mon[41073]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:21:43.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:43 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/819676780' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-10T18:21:43.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:43 smithi115 ceph-mon[41073]: osdmap e52: 8 total, 8 up, 8 in 2024-09-10T18:21:43.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:43 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:43.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:43 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:43.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:43 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:21:43.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:43 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:21:43.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:43 smithi115 ceph-mon[29603]: pgmap v171: 36 pgs: 5 creating+peering, 30 unknown, 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T18:21:43.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:43 smithi115 ceph-mon[29603]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:21:43.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:43 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/819676780' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-10T18:21:43.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:43 smithi115 ceph-mon[29603]: osdmap e52: 8 total, 8 up, 8 in 2024-09-10T18:21:43.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:43 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:43.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:43 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:43.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:43 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:21:43.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:43 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:21:44.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:44 smithi115 ceph-mon[29603]: Checking dashboard <-> RGW credentials 2024-09-10T18:21:44.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:44 smithi115 ceph-mon[29603]: osdmap e53: 8 total, 8 up, 8 in 2024-09-10T18:21:44.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:44 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1588505786' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:21:44.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:44 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/819676780' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:21:44.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:44 smithi115 ceph-mon[29603]: from='client.? 172.21.15.150:0/1650235009' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:21:44.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:44 smithi115 ceph-mon[29603]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:21:44.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:44 smithi115 ceph-mon[41073]: Checking dashboard <-> RGW credentials 2024-09-10T18:21:44.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:44 smithi115 ceph-mon[41073]: osdmap e53: 8 total, 8 up, 8 in 2024-09-10T18:21:44.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:44 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1588505786' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:21:44.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:44 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/819676780' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:21:44.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:44 smithi115 ceph-mon[41073]: from='client.? 172.21.15.150:0/1650235009' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:21:44.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:44 smithi115 ceph-mon[41073]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:21:44.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:44 smithi150 ceph-mon[32761]: Checking dashboard <-> RGW credentials 2024-09-10T18:21:44.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:44 smithi150 ceph-mon[32761]: osdmap e53: 8 total, 8 up, 8 in 2024-09-10T18:21:44.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:44 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1588505786' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:21:44.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:44 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/819676780' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:21:44.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:44 smithi150 ceph-mon[32761]: from='client.? 172.21.15.150:0/1650235009' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:21:44.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:44 smithi150 ceph-mon[32761]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:21:45.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:45 smithi115 ceph-mon[41073]: pgmap v174: 68 pgs: 14 creating+peering, 38 unknown, 16 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:21:45.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:45 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1588505786' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:21:45.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:45 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/819676780' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:21:45.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:45 smithi115 ceph-mon[41073]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-10T18:21:45.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:45 smithi115 ceph-mon[41073]: osdmap e54: 8 total, 8 up, 8 in 2024-09-10T18:21:45.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:45 smithi115 ceph-mon[29603]: pgmap v174: 68 pgs: 14 creating+peering, 38 unknown, 16 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:21:45.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:45 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1588505786' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:21:45.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:45 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/819676780' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:21:45.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:45 smithi115 ceph-mon[29603]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-10T18:21:45.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:45 smithi115 ceph-mon[29603]: osdmap e54: 8 total, 8 up, 8 in 2024-09-10T18:21:45.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:45 smithi150 ceph-mon[32761]: pgmap v174: 68 pgs: 14 creating+peering, 38 unknown, 16 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:21:45.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:45 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1588505786' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:21:45.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:45 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/819676780' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:21:45.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:45 smithi150 ceph-mon[32761]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-10T18:21:45.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:45 smithi150 ceph-mon[32761]: osdmap e54: 8 total, 8 up, 8 in 2024-09-10T18:21:46.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:46 smithi115 ceph-mon[29603]: osdmap e55: 8 total, 8 up, 8 in 2024-09-10T18:21:46.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:46 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1588505786' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:21:46.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:46 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/819676780' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:21:46.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:46 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1588505786' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:21:46.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:46 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/819676780' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:21:46.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:46 smithi115 ceph-mon[29603]: osdmap e56: 8 total, 8 up, 8 in 2024-09-10T18:21:46.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:46 smithi115 ceph-mon[41073]: osdmap e55: 8 total, 8 up, 8 in 2024-09-10T18:21:46.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:46 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1588505786' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:21:46.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:46 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/819676780' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:21:46.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:46 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1588505786' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:21:46.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:46 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/819676780' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:21:46.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:46 smithi115 ceph-mon[41073]: osdmap e56: 8 total, 8 up, 8 in 2024-09-10T18:21:46.554 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:46 smithi150 ceph-mon[32761]: osdmap e55: 8 total, 8 up, 8 in 2024-09-10T18:21:46.554 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:46 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1588505786' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:21:46.554 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:46 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/819676780' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:21:46.554 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:46 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1588505786' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:21:46.554 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:46 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/819676780' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:21:46.554 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:46 smithi150 ceph-mon[32761]: osdmap e56: 8 total, 8 up, 8 in 2024-09-10T18:21:46.815 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.150 --placement '1;smithi150=iscsi.a' 2024-09-10T18:21:47.059 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:21:47.423 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:47 smithi150 ceph-mon[32761]: pgmap v177: 100 pgs: 21 creating+peering, 49 unknown, 30 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.5 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-10T18:21:47.424 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:47 smithi150 ceph-mon[32761]: osdmap e57: 8 total, 8 up, 8 in 2024-09-10T18:21:47.424 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:47 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/563183760' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:21:47.424 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:47 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/3930316940' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:21:47.424 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:47 smithi150 ceph-mon[32761]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:21:47.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:47 smithi115 ceph-mon[41073]: pgmap v177: 100 pgs: 21 creating+peering, 49 unknown, 30 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.5 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-10T18:21:47.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:47 smithi115 ceph-mon[41073]: osdmap e57: 8 total, 8 up, 8 in 2024-09-10T18:21:47.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:47 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/563183760' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:21:47.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:47 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/3930316940' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:21:47.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:47 smithi115 ceph-mon[41073]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:21:47.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:47 smithi115 ceph-mon[29603]: pgmap v177: 100 pgs: 21 creating+peering, 49 unknown, 30 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.5 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-10T18:21:47.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:47 smithi115 ceph-mon[29603]: osdmap e57: 8 total, 8 up, 8 in 2024-09-10T18:21:47.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:47 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/563183760' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:21:47.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:47 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/3930316940' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:21:47.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:47 smithi115 ceph-mon[29603]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:21:48.836 INFO:teuthology.orchestra.run.smithi150.stdout:Scheduled iscsi.datapool update... 2024-09-10T18:21:49.393 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:49 smithi150 ceph-mon[32761]: pgmap v180: 132 pgs: 22 creating+peering, 25 unknown, 85 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-10T18:21:49.394 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:49 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/563183760' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:21:49.394 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:49 smithi150 ceph-mon[32761]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:21:49.394 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:49 smithi150 ceph-mon[32761]: osdmap e58: 8 total, 8 up, 8 in 2024-09-10T18:21:49.394 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:49 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/563183760' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:21:49.394 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:49 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/3930316940' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:21:49.394 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:49 smithi150 ceph-mon[32761]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:21:49.394 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:49 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:49.420 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-09-10T18:21:49.420 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T18:21:49.420 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-10T18:21:49.450 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-10T18:21:49.450 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-10T18:21:49.478 DEBUG:teuthology.orchestra.run.smithi150:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-a600e180-6fa0-11ef-bcea-c7b262605968@iscsi.iscsi.a.service 2024-09-10T18:21:49.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:49 smithi115 ceph-mon[41073]: pgmap v180: 132 pgs: 22 creating+peering, 25 unknown, 85 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-10T18:21:49.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:49 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/563183760' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:21:49.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:49 smithi115 ceph-mon[41073]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:21:49.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:49 smithi115 ceph-mon[41073]: osdmap e58: 8 total, 8 up, 8 in 2024-09-10T18:21:49.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:49 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/563183760' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:21:49.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:49 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/3930316940' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:21:49.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:49 smithi115 ceph-mon[41073]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:21:49.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:49 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:49.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:49 smithi115 ceph-mon[29603]: pgmap v180: 132 pgs: 22 creating+peering, 25 unknown, 85 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-10T18:21:49.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:49 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/563183760' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:21:49.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:49 smithi115 ceph-mon[29603]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:21:49.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:49 smithi115 ceph-mon[29603]: osdmap e58: 8 total, 8 up, 8 in 2024-09-10T18:21:49.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:49 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/563183760' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:21:49.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:49 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/3930316940' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:21:49.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:49 smithi115 ceph-mon[29603]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:21:49.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:49 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:49.521 INFO:tasks.cephadm:Adding prometheus.a on smithi150 2024-09-10T18:21:49.521 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph orch apply prometheus '1;smithi150=a' 2024-09-10T18:21:50.445 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:21:50.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:50 smithi115 ceph-mon[41073]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.150", "placement": "1;smithi150=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:21:50.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:50 smithi115 ceph-mon[41073]: Saving service iscsi.datapool spec with placement smithi150=iscsi.a;count:1 2024-09-10T18:21:50.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:50 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/563183760' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:21:50.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:50 smithi115 ceph-mon[41073]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:21:50.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:50 smithi115 ceph-mon[41073]: osdmap e59: 8 total, 8 up, 8 in 2024-09-10T18:21:50.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:50 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:50.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:50 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:50.487 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:50 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:50.487 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:50 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-10T18:21:50.487 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:50 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-10T18:21:50.487 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:50 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:21:50.487 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:50 smithi115 ceph-mon[41073]: Deploying daemon iscsi.iscsi.a on smithi150 2024-09-10T18:21:50.488 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:50 smithi115 ceph-mon[29603]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.150", "placement": "1;smithi150=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:21:50.488 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:50 smithi115 ceph-mon[29603]: Saving service iscsi.datapool spec with placement smithi150=iscsi.a;count:1 2024-09-10T18:21:50.488 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:50 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/563183760' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:21:50.488 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:50 smithi115 ceph-mon[29603]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:21:50.488 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:50 smithi115 ceph-mon[29603]: osdmap e59: 8 total, 8 up, 8 in 2024-09-10T18:21:50.488 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:50 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:50.488 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:50 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:50.489 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:50 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:50.489 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:50 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-10T18:21:50.489 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:50 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-10T18:21:50.489 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:50 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:21:50.489 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:50 smithi115 ceph-mon[29603]: Deploying daemon iscsi.iscsi.a on smithi150 2024-09-10T18:21:50.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:50 smithi150 ceph-mon[32761]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.150", "placement": "1;smithi150=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:21:50.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:50 smithi150 ceph-mon[32761]: Saving service iscsi.datapool spec with placement smithi150=iscsi.a;count:1 2024-09-10T18:21:50.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:50 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/563183760' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:21:50.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:50 smithi150 ceph-mon[32761]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:21:50.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:50 smithi150 ceph-mon[32761]: osdmap e59: 8 total, 8 up, 8 in 2024-09-10T18:21:50.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:50 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:50.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:50 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:50.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:50 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:50.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:50 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-10T18:21:50.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:50 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-10T18:21:50.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:50 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:21:50.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:50 smithi150 ceph-mon[32761]: Deploying daemon iscsi.iscsi.a on smithi150 2024-09-10T18:21:51.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:51 smithi115 ceph-mon[29603]: pgmap v183: 132 pgs: 17 creating+peering, 17 unknown, 98 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:21:51.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:51 smithi115 ceph-mon[29603]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T18:21:51.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:51 smithi115 ceph-mon[29603]: Cluster is now healthy 2024-09-10T18:21:51.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:51 smithi115 ceph-mon[41073]: pgmap v183: 132 pgs: 17 creating+peering, 17 unknown, 98 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:21:51.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:51 smithi115 ceph-mon[41073]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T18:21:51.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:51 smithi115 ceph-mon[41073]: Cluster is now healthy 2024-09-10T18:21:51.528 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:51 smithi150 ceph-mon[32761]: pgmap v183: 132 pgs: 17 creating+peering, 17 unknown, 98 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:21:51.528 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:51 smithi150 ceph-mon[32761]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T18:21:51.528 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:51 smithi150 ceph-mon[32761]: Cluster is now healthy 2024-09-10T18:21:52.727 INFO:teuthology.orchestra.run.smithi150.stdout:Scheduled prometheus update... 2024-09-10T18:21:53.277 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:53 smithi150 ceph-mon[32761]: osdmap e60: 8 total, 8 up, 8 in 2024-09-10T18:21:53.277 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:53 smithi150 ceph-mon[32761]: pgmap v185: 132 pgs: 10 creating+peering, 7 unknown, 115 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 1.5 KiB/s wr, 41 op/s 2024-09-10T18:21:53.277 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:53 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:53.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:53 smithi115 ceph-mon[29603]: osdmap e60: 8 total, 8 up, 8 in 2024-09-10T18:21:53.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:53 smithi115 ceph-mon[29603]: pgmap v185: 132 pgs: 10 creating+peering, 7 unknown, 115 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 1.5 KiB/s wr, 41 op/s 2024-09-10T18:21:53.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:53 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:53.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:53 smithi115 ceph-mon[41073]: osdmap e60: 8 total, 8 up, 8 in 2024-09-10T18:21:53.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:53 smithi115 ceph-mon[41073]: pgmap v185: 132 pgs: 10 creating+peering, 7 unknown, 115 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 1.5 KiB/s wr, 41 op/s 2024-09-10T18:21:53.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:53 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:53.548 DEBUG:teuthology.orchestra.run.smithi150:prometheus.a> sudo journalctl -f -n 0 -u ceph-a600e180-6fa0-11ef-bcea-c7b262605968@prometheus.a.service 2024-09-10T18:21:53.550 INFO:tasks.cephadm:Adding node-exporter.a on smithi115 2024-09-10T18:21:53.550 INFO:tasks.cephadm:Adding node-exporter.b on smithi150 2024-09-10T18:21:53.551 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph orch apply node-exporter '2;smithi115=a;smithi150=b' 2024-09-10T18:21:53.805 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:21:53 smithi150 bash[61978]: c03d5be2b109152142e55d2bc9822e052602b3a7e998b2dd12edecfd15ac2ffc 2024-09-10T18:21:53.837 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:21:54.275 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:54 smithi150 ceph-mon[32761]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi150=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:21:54.275 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:54 smithi150 ceph-mon[32761]: Saving service prometheus spec with placement smithi150=a;count:1 2024-09-10T18:21:54.275 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:21:53 smithi150 podman[62162]: 2024-09-10 18:21:53.836322735 +0000 UTC m=+0.017091811 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:21:54.275 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:21:53 smithi150 podman[62162]: 2024-09-10 18:21:53.941559452 +0000 UTC m=+0.122328573 container create 8aed98e74f71fceaa186e06c34dc1b969eccbff5c2f348c12af84787e9c2cb20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-10T18:21:54.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:54 smithi115 ceph-mon[29603]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi150=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:21:54.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:54 smithi115 ceph-mon[29603]: Saving service prometheus spec with placement smithi150=a;count:1 2024-09-10T18:21:54.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:54 smithi115 ceph-mon[41073]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi150=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:21:54.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:54 smithi115 ceph-mon[41073]: Saving service prometheus spec with placement smithi150=a;count:1 2024-09-10T18:21:54.549 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:21:54 smithi150 podman[62162]: 2024-09-10 18:21:54.379882864 +0000 UTC m=+0.560651939 container init 8aed98e74f71fceaa186e06c34dc1b969eccbff5c2f348c12af84787e9c2cb20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:21:54.549 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:21:54 smithi150 podman[62162]: 2024-09-10 18:21:54.385340021 +0000 UTC m=+0.566109098 container start 8aed98e74f71fceaa186e06c34dc1b969eccbff5c2f348c12af84787e9c2cb20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-10T18:21:54.550 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:21:54 smithi150 bash[62162]: 8aed98e74f71fceaa186e06c34dc1b969eccbff5c2f348c12af84787e9c2cb20 2024-09-10T18:21:54.550 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:21:54 smithi150 systemd[1]: Started Ceph iscsi.iscsi.a for a600e180-6fa0-11ef-bcea-c7b262605968. 2024-09-10T18:21:54.805 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:21:54 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug Started the configuration object watcher 2024-09-10T18:21:54.805 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:21:54 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug Checking for config object changes every 1s 2024-09-10T18:21:54.805 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:21:54 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug Processing osd blocklist entries for this node 2024-09-10T18:21:55.305 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:21:54 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug Reading the configuration object to update local LIO configuration 2024-09-10T18:21:55.305 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:21:54 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug Configuration does not have an entry for this host(smithi150.front.sepia.ceph.com) - nothing to define to LIO 2024-09-10T18:21:55.305 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:21:54 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: * Serving Flask app 'rbd-target-api' (lazy loading) 2024-09-10T18:21:55.305 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:21:54 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: * Environment: production 2024-09-10T18:21:55.306 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:21:54 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-10T18:21:55.306 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:21:54 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: Use a production WSGI server instead. 2024-09-10T18:21:55.306 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:21:54 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: * Debug mode: off 2024-09-10T18:21:55.306 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:21:54 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug * Running on all addresses. 2024-09-10T18:21:55.306 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:21:54 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-10T18:21:55.306 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:21:54 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: * Running on all addresses. 2024-09-10T18:21:55.306 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:21:54 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-10T18:21:55.306 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:21:54 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug * Running on http://[fe80::ec4:7aff:fe88:6e35]:5000/ (Press CTRL+C to quit) 2024-09-10T18:21:55.306 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:21:54 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: * Running on http://[fe80::ec4:7aff:fe88:6e35]:5000/ (Press CTRL+C to quit) 2024-09-10T18:21:55.656 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:55 smithi150 ceph-mon[32761]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 77 KiB/s rd, 5.8 KiB/s wr, 186 op/s 2024-09-10T18:21:55.656 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:55 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:55.656 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:55 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:55.656 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:55 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:55.656 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:55 smithi150 ceph-mon[32761]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-10T18:21:55.656 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:55 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:55.656 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:55 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:21:55.656 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:55 smithi150 ceph-mon[32761]: from='client.? 172.21.15.150:0/1360807431' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T18:21:55.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:55 smithi115 ceph-mon[29603]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 77 KiB/s rd, 5.8 KiB/s wr, 186 op/s 2024-09-10T18:21:55.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:55 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:55.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:55 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:55.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:55 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:55.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:55 smithi115 ceph-mon[29603]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-10T18:21:55.986 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:55 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:55.986 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:55 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:21:55.986 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:55 smithi115 ceph-mon[29603]: from='client.? 172.21.15.150:0/1360807431' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T18:21:55.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:55 smithi115 ceph-mon[41073]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 77 KiB/s rd, 5.8 KiB/s wr, 186 op/s 2024-09-10T18:21:55.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:55 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:55.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:55 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:55.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:55 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:55.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:55 smithi115 ceph-mon[41073]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-10T18:21:55.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:55 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:55.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:55 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:21:55.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:55 smithi115 ceph-mon[41073]: from='client.? 172.21.15.150:0/1360807431' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T18:21:56.050 INFO:teuthology.orchestra.run.smithi150.stdout:Scheduled node-exporter update... 2024-09-10T18:21:57.179 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:57 smithi150 ceph-mon[32761]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 59 KiB/s rd, 4.5 KiB/s wr, 143 op/s 2024-09-10T18:21:57.180 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:57 smithi150 ceph-mon[32761]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi115=a;smithi150=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:21:57.180 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:57 smithi150 ceph-mon[32761]: Saving service node-exporter spec with placement smithi115=a;smithi150=b;count:2 2024-09-10T18:21:57.180 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:57 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:57.207 DEBUG:teuthology.orchestra.run.smithi115:node-exporter.a> sudo journalctl -f -n 0 -u ceph-a600e180-6fa0-11ef-bcea-c7b262605968@node-exporter.a.service 2024-09-10T18:21:57.210 DEBUG:teuthology.orchestra.run.smithi150:node-exporter.b> sudo journalctl -f -n 0 -u ceph-a600e180-6fa0-11ef-bcea-c7b262605968@node-exporter.b.service 2024-09-10T18:21:57.213 INFO:tasks.cephadm:Adding alertmanager.a on smithi115 2024-09-10T18:21:57.213 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph orch apply alertmanager '1;smithi115=a' 2024-09-10T18:21:57.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:57 smithi115 ceph-mon[41073]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 59 KiB/s rd, 4.5 KiB/s wr, 143 op/s 2024-09-10T18:21:57.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:57 smithi115 ceph-mon[41073]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi115=a;smithi150=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:21:57.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:57 smithi115 ceph-mon[41073]: Saving service node-exporter spec with placement smithi115=a;smithi150=b;count:2 2024-09-10T18:21:57.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:57 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:57.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:57 smithi115 ceph-mon[29603]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 59 KiB/s rd, 4.5 KiB/s wr, 143 op/s 2024-09-10T18:21:57.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:57 smithi115 ceph-mon[29603]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi115=a;smithi150=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:21:57.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:57 smithi115 ceph-mon[29603]: Saving service node-exporter spec with placement smithi115=a;smithi150=b;count:2 2024-09-10T18:21:57.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:57 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:57.979 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:21:58.395 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:58 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:58.396 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:58 smithi150 ceph-mon[32761]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-10T18:21:58.396 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:58 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:58.396 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:58 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:58.396 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:58 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:21:58.396 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:58 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:21:58.396 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:58 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:58.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:58 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:58.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:58 smithi115 ceph-mon[41073]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-10T18:21:58.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:58 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:58.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:58 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:58.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:58 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:21:58.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:58 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:21:58.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:58 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:58.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:58 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:58.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:58 smithi115 ceph-mon[29603]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-10T18:21:58.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:58 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:58.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:58 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:58.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:58 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:21:58.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:58 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:21:58.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:58 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:21:59.267 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:59 smithi150 ceph-mon[32761]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 53 KiB/s rd, 4.1 KiB/s wr, 128 op/s 2024-09-10T18:21:59.267 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:21:59 smithi150 ceph-mon[32761]: Deploying daemon prometheus.a on smithi150 2024-09-10T18:21:59.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:59 smithi115 ceph-mon[41073]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 53 KiB/s rd, 4.1 KiB/s wr, 128 op/s 2024-09-10T18:21:59.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:21:59 smithi115 ceph-mon[41073]: Deploying daemon prometheus.a on smithi150 2024-09-10T18:21:59.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:59 smithi115 ceph-mon[29603]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 53 KiB/s rd, 4.1 KiB/s wr, 128 op/s 2024-09-10T18:21:59.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:21:59 smithi115 ceph-mon[29603]: Deploying daemon prometheus.a on smithi150 2024-09-10T18:21:59.763 INFO:teuthology.orchestra.run.smithi150.stdout:Scheduled alertmanager update... 2024-09-10T18:22:00.702 DEBUG:teuthology.orchestra.run.smithi115:alertmanager.a> sudo journalctl -f -n 0 -u ceph-a600e180-6fa0-11ef-bcea-c7b262605968@alertmanager.a.service 2024-09-10T18:22:00.705 INFO:tasks.cephadm:Adding grafana.a on smithi150 2024-09-10T18:22:00.705 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph orch apply grafana '1;smithi150=a' 2024-09-10T18:22:00.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:00 smithi115 ceph-mon[41073]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi115=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:22:00.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:00 smithi115 ceph-mon[41073]: Saving service alertmanager spec with placement smithi115=a;count:1 2024-09-10T18:22:00.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:00 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:22:00.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:00 smithi115 ceph-mon[41073]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 46 KiB/s rd, 3.6 KiB/s wr, 113 op/s 2024-09-10T18:22:00.986 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:00 smithi115 ceph-mon[29603]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi115=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:22:00.986 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:00 smithi115 ceph-mon[29603]: Saving service alertmanager spec with placement smithi115=a;count:1 2024-09-10T18:22:00.986 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:00 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:22:00.986 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:00 smithi115 ceph-mon[29603]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 46 KiB/s rd, 3.6 KiB/s wr, 113 op/s 2024-09-10T18:22:00.997 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:22:01.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:00 smithi150 ceph-mon[32761]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi115=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:22:01.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:00 smithi150 ceph-mon[32761]: Saving service alertmanager spec with placement smithi115=a;count:1 2024-09-10T18:22:01.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:00 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:22:01.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:00 smithi150 ceph-mon[32761]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 46 KiB/s rd, 3.6 KiB/s wr, 113 op/s 2024-09-10T18:22:03.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:03 smithi150 ceph-mon[32761]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 38 KiB/s rd, 3.0 KiB/s wr, 93 op/s 2024-09-10T18:22:03.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:03 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:22:03.451 INFO:teuthology.orchestra.run.smithi150.stdout:Scheduled grafana update... 2024-09-10T18:22:03.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:03 smithi115 ceph-mon[41073]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 38 KiB/s rd, 3.0 KiB/s wr, 93 op/s 2024-09-10T18:22:03.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:03 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:22:03.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:03 smithi115 ceph-mon[29603]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 38 KiB/s rd, 3.0 KiB/s wr, 93 op/s 2024-09-10T18:22:03.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:03 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:22:04.629 DEBUG:teuthology.orchestra.run.smithi150:grafana.a> sudo journalctl -f -n 0 -u ceph-a600e180-6fa0-11ef-bcea-c7b262605968@grafana.a.service 2024-09-10T18:22:04.631 INFO:tasks.cephadm:Setting up client nodes... 2024-09-10T18:22:04.632 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T18:22:04.713 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:04 smithi150 ceph-mon[32761]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi150=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:22:04.713 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:04 smithi150 ceph-mon[32761]: Saving service grafana spec with placement smithi150=a;count:1 2024-09-10T18:22:04.713 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:04 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:22:04.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:04 smithi115 ceph-mon[29603]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi150=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:22:04.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:04 smithi115 ceph-mon[29603]: Saving service grafana spec with placement smithi150=a;count:1 2024-09-10T18:22:04.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:04 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:22:04.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:04 smithi115 ceph-mon[41073]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi150=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:22:04.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:04 smithi115 ceph-mon[41073]: Saving service grafana spec with placement smithi150=a;count:1 2024-09-10T18:22:04.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:04 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:22:04.886 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:22:05.055 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:22:04 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:22:05.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:05 smithi115 ceph-mon[29603]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 33 KiB/s rd, 2.5 KiB/s wr, 78 op/s 2024-09-10T18:22:05.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:05 smithi115 ceph-mon[41073]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 33 KiB/s rd, 2.5 KiB/s wr, 78 op/s 2024-09-10T18:22:05.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:05 smithi150 ceph-mon[32761]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 33 KiB/s rd, 2.5 KiB/s wr, 78 op/s 2024-09-10T18:22:06.733 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:06 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:22:06.733 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:06 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:22:06.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:06 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:22:06.938 INFO:teuthology.orchestra.run.smithi115.stdout:[client.0] 2024-09-10T18:22:06.938 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBOjuBmygSdNxAAJh2ggfvK0XKtmM3QIBa3hA== 2024-09-10T18:22:07.586 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:07 smithi115 ceph-mon[41073]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:22:07.586 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:07 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/2869381181' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:22:07.586 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:07 smithi115 ceph-mon[41073]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:22:07.586 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:07 smithi115 ceph-mon[41073]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:22:07.587 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:07 smithi115 ceph-mon[29603]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:22:07.587 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:07 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/2869381181' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:22:07.587 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:07 smithi115 ceph-mon[29603]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:22:07.587 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:07 smithi115 ceph-mon[29603]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:22:07.610 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T18:22:07.611 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-10T18:22:07.611 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T18:22:07.648 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T18:22:07.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:07 smithi150 ceph-mon[32761]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:22:07.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:07 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/2869381181' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:22:07.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:07 smithi150 ceph-mon[32761]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:22:07.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:07 smithi150 ceph-mon[32761]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:22:08.887 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:22:09.552 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:09 smithi150 ceph-mon[32761]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:22:09.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:09 smithi115 ceph-mon[41073]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:22:09.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:09 smithi115 ceph-mon[29603]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:22:11.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:11 smithi115 ceph-mon[41073]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:22:11.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:11 smithi115 ceph-mon[29603]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:22:11.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:11 smithi150 ceph-mon[32761]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:22:11.805 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:22:11 smithi150 systemd[1]: Starting Ceph prometheus.a for a600e180-6fa0-11ef-bcea-c7b262605968... 2024-09-10T18:22:12.591 INFO:teuthology.orchestra.run.smithi150.stdout:[client.1] 2024-09-10T18:22:12.591 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBUjuBmPYrJIhAAh8ArURIgtKCWNJVdGKTvyQ== 2024-09-10T18:22:12.805 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:22:12 smithi150 podman[64017]: 2024-09-10 18:22:12.380042261 +0000 UTC m=+0.017069918 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-10T18:22:12.805 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:22:12 smithi150 podman[64017]: 2024-09-10 18:22:12.497173429 +0000 UTC m=+0.134201086 container create f58a458762247727188bf35cb7c96f9b8305576ba3c456ef2181b06971bf1a6c (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:22:13.208 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-10T18:22:13.208 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-10T18:22:13.209 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-10T18:22:13.247 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T18:22:13.247 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-10T18:22:13.247 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-10T18:22:13.445 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:22:13 smithi150 podman[64017]: 2024-09-10 18:22:13.351869976 +0000 UTC m=+0.988897632 container init f58a458762247727188bf35cb7c96f9b8305576ba3c456ef2181b06971bf1a6c (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:22:13.445 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:22:13 smithi150 podman[64017]: 2024-09-10 18:22:13.357250031 +0000 UTC m=+0.994277690 container start f58a458762247727188bf35cb7c96f9b8305576ba3c456ef2181b06971bf1a6c (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:22:13.445 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:22:13 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[64035]: ts=2024-09-10T18:22:13.380Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-10T18:22:13.445 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:22:13 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[64035]: ts=2024-09-10T18:22:13.380Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-10T18:22:13.445 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:22:13 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[64035]: ts=2024-09-10T18:22:13.380Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi150 (none))" 2024-09-10T18:22:13.445 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:22:13 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[64035]: ts=2024-09-10T18:22:13.380Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-10T18:22:13.445 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:22:13 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[64035]: ts=2024-09-10T18:22:13.380Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-10T18:22:13.446 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:22:13 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[64035]: ts=2024-09-10T18:22:13.381Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-10T18:22:13.446 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:22:13 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[64035]: ts=2024-09-10T18:22:13.381Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-10T18:22:13.446 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:22:13 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[64035]: ts=2024-09-10T18:22:13.383Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-10T18:22:13.446 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:22:13 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[64035]: ts=2024-09-10T18:22:13.383Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-10T18:22:13.446 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:22:13 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[64035]: ts=2024-09-10T18:22:13.385Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-10T18:22:13.446 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:22:13 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[64035]: ts=2024-09-10T18:22:13.385Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.591µs 2024-09-10T18:22:13.446 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:22:13 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[64035]: ts=2024-09-10T18:22:13.385Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-10T18:22:13.446 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:22:13 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[64035]: ts=2024-09-10T18:22:13.385Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-09-10T18:22:13.446 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:22:13 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[64035]: ts=2024-09-10T18:22:13.385Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=17.698µs wal_replay_duration=172.685µs wbl_replay_duration=122ns total_replay_duration=208.458µs 2024-09-10T18:22:13.446 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:22:13 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[64035]: ts=2024-09-10T18:22:13.386Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-10T18:22:13.446 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:22:13 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[64035]: ts=2024-09-10T18:22:13.386Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-10T18:22:13.446 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:22:13 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[64035]: ts=2024-09-10T18:22:13.386Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-10T18:22:13.446 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:22:13 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[64035]: ts=2024-09-10T18:22:13.400Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=14.7448ms db_storage=720ns remote_storage=1.512µs web_handler=386ns query_engine=436ns scrape=447.129µs scrape_sd=16.981µs notify=1.223µs notify_sd=1.548µs rules=13.937554ms tracing=5.965µs 2024-09-10T18:22:13.447 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:22:13 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[64035]: ts=2024-09-10T18:22:13.400Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-10T18:22:13.447 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:22:13 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[64035]: ts=2024-09-10T18:22:13.400Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-10T18:22:13.447 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:22:13 smithi150 bash[64017]: f58a458762247727188bf35cb7c96f9b8305576ba3c456ef2181b06971bf1a6c 2024-09-10T18:22:13.502 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:22:13.625 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:13 smithi115 ceph-mon[41073]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:22:13.625 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:13 smithi115 ceph-mon[41073]: from='client.? 172.21.15.150:0/3157898656' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:22:13.628 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:13 smithi115 ceph-mon[41073]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:22:13.628 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:13 smithi115 ceph-mon[41073]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:22:13.628 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:13 smithi115 ceph-mon[29603]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:22:13.628 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:13 smithi115 ceph-mon[29603]: from='client.? 172.21.15.150:0/3157898656' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:22:13.629 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:13 smithi115 ceph-mon[29603]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:22:13.629 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:13 smithi115 ceph-mon[29603]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:22:13.805 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:22:13 smithi150 systemd[1]: Started Ceph prometheus.a for a600e180-6fa0-11ef-bcea-c7b262605968. 2024-09-10T18:22:13.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:13 smithi150 ceph-mon[32761]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:22:13.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:13 smithi150 ceph-mon[32761]: from='client.? 172.21.15.150:0/3157898656' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:22:13.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:13 smithi150 ceph-mon[32761]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:22:13.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:13 smithi150 ceph-mon[32761]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:22:14.660 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:14 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ignoring --setuser ceph since I am not root 2024-09-10T18:22:14.660 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:14 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ignoring --setgroup ceph since I am not root 2024-09-10T18:22:14.660 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:14 smithi115 ceph-mgr[29877]: -- 172.21.15.115:0/3995257421 <== mon.2 v2:172.21.15.115:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5601f9680340 con 0x5601f968cc00 2024-09-10T18:22:14.661 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:14 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:22:14.661 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:14 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:22:14.661 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:14 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:22:14.661 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:14 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-10T18:22:14.661 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:14 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:22:14.661 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:14 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:22:14.661 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:14 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:22:14.661 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:14 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-10T18:22:14.783 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:14 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:22:14.783 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:14 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:22:14.783 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:14 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' 2024-09-10T18:22:14.783 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:14 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-10T18:22:14.783 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ignoring --setuser ceph since I am not root 2024-09-10T18:22:14.783 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ignoring --setgroup ceph since I am not root 2024-09-10T18:22:14.784 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:14 smithi150 ceph-mgr[34838]: -- 172.21.15.150:0/581136156 <== mon.0 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559d79cc4340 con 0x559d79cd0800 2024-09-10T18:22:14.784 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:22:14.661+0000 7fc770dafec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:22:14.784 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:22:14.715+0000 7fc770dafec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:22:14.784 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:22:14.780+0000 7fc770dafec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:22:14.917 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:14 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:22:14.658+0000 7faec26c6ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:22:14.917 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:14 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:22:14.714+0000 7faec26c6ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:22:14.917 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:14 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:22:14.780+0000 7faec26c6ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:22:14.917 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:14 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:22:14.915+0000 7faec26c6ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:22:15.037 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:22:14.901+0000 7fc770dafec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:22:15.196 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:15 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:22:15.052+0000 7faec26c6ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:22:15.291 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:15 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:22:15.035+0000 7fc770dafec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:22:15.291 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:15 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:22:15.288+0000 7fc770dafec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:22:15.485 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:15 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:22:15.320+0000 7faec26c6ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:22:15.485 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:15 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:22:15.382+0000 7faec26c6ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:22:15.492 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:22:15.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:15 smithi150 ceph-mon[32761]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-10T18:22:15.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:15 smithi150 ceph-mon[32761]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-10T18:22:15.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:15 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/2363532919' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T18:22:15.555 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:15 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:22:15.348+0000 7fc770dafec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:22:15.555 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:15 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:22:15.508+0000 7fc770dafec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:22:15.817 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:15 smithi115 ceph-mon[41073]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-10T18:22:15.817 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:15 smithi115 ceph-mon[41073]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-10T18:22:15.817 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:15 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/2363532919' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T18:22:15.817 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:15 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:22:15.550+0000 7faec26c6ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:22:15.817 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:15 smithi115 ceph-mon[29603]: from='mgr.14150 172.21.15.115:0/2148167426' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-10T18:22:15.817 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:15 smithi115 ceph-mon[29603]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-10T18:22:15.817 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:15 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/2363532919' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T18:22:16.250 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":667858637}]},"active_addr":"172.21.15.115:6800/667858637","active_change":"2024-09-10T18:17:21.978020+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":24106,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.115:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.115:0","nonce":570209858}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.115:0","nonce":4136881386}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.115:0","nonce":337878570}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.115:0","nonce":171922759}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.115:0","nonce":498921679}]}]} 2024-09-10T18:22:16.256 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-10T18:22:16.257 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-10T18:22:16.257 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T18:22:16.479 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:16 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:22:16.406+0000 7faec26c6ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:22:16.480 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:16 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:22:16.477+0000 7faec26c6ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:22:16.506 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:22:16.805 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:16 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:22:16.344+0000 7fc770dafec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:22:16.805 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:16 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:22:16.407+0000 7fc770dafec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:22:17.315 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:17 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:22:17.201+0000 7faec26c6ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:22:17.388 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:17 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:22:17.100+0000 7fc770dafec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:22:17.388 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:17 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:22:17.209+0000 7fc770dafec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:22:17.388 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:17 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:22:17.326+0000 7fc770dafec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:22:17.621 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:17 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:22:17.312+0000 7faec26c6ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:22:17.621 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:17 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:22:17.435+0000 7faec26c6ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:22:17.621 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:17 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:22:17.494+0000 7faec26c6ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:22:17.621 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:17 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:22:17.619+0000 7faec26c6ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:22:17.771 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:17 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:22:17.385+0000 7fc770dafec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:22:17.771 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:17 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:22:17.510+0000 7fc770dafec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:22:17.771 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:17 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:22:17.578+0000 7fc770dafec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:22:17.771 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:17 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:22:17.635+0000 7fc770dafec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:22:17.771 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:17 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:22:17.768+0000 7fc770dafec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:22:17.888 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:17 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:22:17.687+0000 7faec26c6ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:22:17.888 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:17 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:22:17.745+0000 7faec26c6ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:22:17.888 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:17 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:22:17.886+0000 7faec26c6ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:22:18.033 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:17 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:22:17.831+0000 7fc770dafec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:22:18.033 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:17 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:22:17.894+0000 7fc770dafec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:22:18.033 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:17 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:22:17.974+0000 7fc770dafec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:22:18.033 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:18 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:22:18.030+0000 7fc770dafec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:22:18.167 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:17 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:22:17.951+0000 7faec26c6ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:22:18.168 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:22:18.017+0000 7faec26c6ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:22:18.168 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:22:18.105+0000 7faec26c6ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:22:18.168 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:22:18.165+0000 7faec26c6ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:22:18.305 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:18 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:22:18.087+0000 7fc770dafec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:22:18.485 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:22:18.226+0000 7faec26c6ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:22:18.490 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:22:18.490 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":60,"fsid":"a600e180-6fa0-11ef-bcea-c7b262605968","created":"2024-09-10T18:16:27.008296+0000","modified":"2024-09-10T18:21:52.000483+0000","last_up_change":"2024-09-10T18:21:29.447953+0000","last_in_change":"2024-09-10T18:21:12.478592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T18:19:32.280261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-10T18:21:40.912985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-10T18:21:41.147358+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":56,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-10T18:21:42.479868+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-10T18:21:44.452268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-10T18:21:46.516202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"107d9598-28cd-4260-9f5d-a3b7886845c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6801","nonce":4123495067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":4123495067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":4123495067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6803","nonce":4123495067}]},"public_addr":"172.21.15.115:6801/4123495067","cluster_addr":"172.21.15.115:6802/4123495067","heartbeat_back_addr":"172.21.15.115:6804/4123495067","heartbeat_front_addr":"172.21.15.115:6803/4123495067","state":["exists","up"]},{"osd":1,"uuid":"c3cfd69f-7300-4b6e-a57b-33eefed950b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6805","nonce":2506968391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2506968391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2506968391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6807","nonce":2506968391}]},"public_addr":"172.21.15.115:6805/2506968391","cluster_addr":"172.21.15.115:6806/2506968391","heartbeat_back_addr":"172.21.15.115:6808/2506968391","heartbeat_front_addr":"172.21.15.115:6807/2506968391","state":["exists","up"]},{"osd":2,"uuid":"9abe6878-817d-4c81-b4b8-db61147c2f94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6809","nonce":343149052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":343149052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":343149052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6811","nonce":343149052}]},"public_addr":"172.21.15.115:6809/343149052","cluster_addr":"172.21.15.115:6810/343149052","heartbeat_back_addr":"172.21.15.115:6812/343149052","heartbeat_front_addr":"172.21.15.115:6811/343149052","state":["exists","up"]},{"osd":3,"uuid":"4fefb92a-fc8d-4a28-9c89-1b7e214c8e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6813","nonce":3162056893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":3162056893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3162056893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":3162056893}]},"public_addr":"172.21.15.115:6813/3162056893","cluster_addr":"172.21.15.115:6814/3162056893","heartbeat_back_addr":"172.21.15.115:6816/3162056893","heartbeat_front_addr":"172.21.15.115:6815/3162056893","state":["exists","up"]},{"osd":4,"uuid":"fa14f44d-1b45-490d-a686-77da88336564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1929277477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6801","nonce":1929277477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6803","nonce":1929277477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1929277477}]},"public_addr":"172.21.15.150:6800/1929277477","cluster_addr":"172.21.15.150:6801/1929277477","heartbeat_back_addr":"172.21.15.150:6803/1929277477","heartbeat_front_addr":"172.21.15.150:6802/1929277477","state":["exists","up"]},{"osd":5,"uuid":"40cabcd5-6874-45d2-bfaf-d737f834e7f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3667323015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6805","nonce":3667323015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6807","nonce":3667323015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3667323015}]},"public_addr":"172.21.15.150:6804/3667323015","cluster_addr":"172.21.15.150:6805/3667323015","heartbeat_back_addr":"172.21.15.150:6807/3667323015","heartbeat_front_addr":"172.21.15.150:6806/3667323015","state":["exists","up"]},{"osd":6,"uuid":"d871feca-36dd-4180-becc-e3e0dd9b4319","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3335870479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3335870479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6811","nonce":3335870479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3335870479}]},"public_addr":"172.21.15.150:6808/3335870479","cluster_addr":"172.21.15.150:6809/3335870479","heartbeat_back_addr":"172.21.15.150:6811/3335870479","heartbeat_front_addr":"172.21.15.150:6810/3335870479","state":["exists","up"]},{"osd":7,"uuid":"0aaa1287-7337-49a5-a832-709aa3980834","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":672367088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":672367088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6815","nonce":672367088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":672367088}]},"public_addr":"172.21.15.150:6812/672367088","cluster_addr":"172.21.15.150:6813/672367088","heartbeat_back_addr":"172.21.15.150:6815/672367088","heartbeat_front_addr":"172.21.15.150:6814/672367088","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:18:41.264707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:19:05.260563+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:19:29.810350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:19:54.613874+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:20:16.294916+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:20:40.099086+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:21:03.703231+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:21:27.515712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/2382563730":"2024-09-11T18:17:21.977918+0000","172.21.15.115:0/3905115041":"2024-09-11T18:17:21.977918+0000","172.21.15.115:6800/3682910233":"2024-09-11T18:17:21.977918+0000","172.21.15.115:6800/2658904986":"2024-09-11T18:16:59.107583+0000","172.21.15.115:0/553409925":"2024-09-11T18:17:21.977918+0000","172.21.15.115:0/1020299194":"2024-09-11T18:16:59.107583+0000","172.21.15.115:0/3979277447":"2024-09-11T18:16:59.107583+0000","172.21.15.115:0/936574235":"2024-09-11T18:16:59.107583+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T18:22:18.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:18 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/2272817258' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:22:18.826 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:18 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/2272817258' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:22:18.826 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:18 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/2272817258' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:22:19.084 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:22:19.016+0000 7faec26c6ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:22:19.191 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-10T18:22:19.191 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T18:22:19.279 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:18 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:22:18.830+0000 7fc770dafec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:22:19.279 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:18 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: [10/Sep/2024:18:22:18] ENGINE Bus STARTING 2024-09-10T18:22:19.279 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:18 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: CherryPy Checker: 2024-09-10T18:22:19.280 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:18 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: The Application mounted at '' has an empty config. 2024-09-10T18:22:19.280 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:18 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:22:19.280 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:18 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: [10/Sep/2024:18:22:18] ENGINE Serving on http://:::9283 2024-09-10T18:22:19.280 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:18 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: [10/Sep/2024:18:22:18] ENGINE Bus STARTED 2024-09-10T18:22:19.372 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: [10/Sep/2024:18:22:19] ENGINE Bus STARTING 2024-09-10T18:22:19.372 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: CherryPy Checker: 2024-09-10T18:22:19.372 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: The Application mounted at '' has an empty config. 2024-09-10T18:22:19.372 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:22:19.373 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: [10/Sep/2024:18:22:19] ENGINE Serving on http://:::9283 2024-09-10T18:22:19.373 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: [10/Sep/2024:18:22:19] ENGINE Bus STARTED 2024-09-10T18:22:19.471 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:22:19.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[41073]: Standby manager daemon x restarted 2024-09-10T18:22:19.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[41073]: Standby manager daemon x started 2024-09-10T18:22:19.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[41073]: from='mgr.? 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:22:19.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[41073]: from='mgr.? 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:22:19.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[41073]: from='mgr.? 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:22:19.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[41073]: from='mgr.? 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:22:19.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[41073]: Active manager daemon y restarted 2024-09-10T18:22:19.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[41073]: Activating manager daemon y 2024-09-10T18:22:19.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[41073]: osdmap e61: 8 total, 8 up, 8 in 2024-09-10T18:22:19.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[41073]: mgrmap e18: y(active, starting, since 0.0101284s), standbys: x 2024-09-10T18:22:19.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:22:19.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:22:19.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:22:19.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:22:19.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:22:19.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:22:19.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:22:19.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:22:19.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:22:19.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:22:19.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:22:19.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:22:19.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:22:19.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:22:19.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:22:19.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:22:19.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[41073]: Manager daemon y is now available 2024-09-10T18:22:19.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:22:19.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:19.738 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[29603]: Standby manager daemon x restarted 2024-09-10T18:22:19.738 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[29603]: Standby manager daemon x started 2024-09-10T18:22:19.738 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[29603]: from='mgr.? 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:22:19.738 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[29603]: from='mgr.? 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:22:19.738 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[29603]: from='mgr.? 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:22:19.738 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[29603]: from='mgr.? 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:22:19.738 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[29603]: Active manager daemon y restarted 2024-09-10T18:22:19.738 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[29603]: Activating manager daemon y 2024-09-10T18:22:19.738 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[29603]: osdmap e61: 8 total, 8 up, 8 in 2024-09-10T18:22:19.739 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[29603]: mgrmap e18: y(active, starting, since 0.0101284s), standbys: x 2024-09-10T18:22:19.739 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:22:19.739 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:22:19.739 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:22:19.739 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:22:19.739 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:22:19.739 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:22:19.739 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:22:19.739 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:22:19.739 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:22:19.739 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:22:19.739 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:22:19.739 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:22:19.740 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:22:19.740 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:22:19.740 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:22:19.740 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:22:19.740 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[29603]: Manager daemon y is now available 2024-09-10T18:22:19.740 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:22:19.740 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:19.740 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:22:19.740 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:22:19.740 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:22:19.740 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:22:19.740 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:22:19.740 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:22:19.741 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:22:19.741 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:22:19.741 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:22:19.741 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:19 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:22:19.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:19 smithi150 ceph-mon[32761]: Standby manager daemon x restarted 2024-09-10T18:22:19.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:19 smithi150 ceph-mon[32761]: Standby manager daemon x started 2024-09-10T18:22:19.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:19 smithi150 ceph-mon[32761]: from='mgr.? 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:22:19.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:19 smithi150 ceph-mon[32761]: from='mgr.? 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:22:19.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:19 smithi150 ceph-mon[32761]: from='mgr.? 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:22:19.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:19 smithi150 ceph-mon[32761]: from='mgr.? 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:22:19.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:19 smithi150 ceph-mon[32761]: Active manager daemon y restarted 2024-09-10T18:22:19.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:19 smithi150 ceph-mon[32761]: Activating manager daemon y 2024-09-10T18:22:19.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:19 smithi150 ceph-mon[32761]: osdmap e61: 8 total, 8 up, 8 in 2024-09-10T18:22:19.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:19 smithi150 ceph-mon[32761]: mgrmap e18: y(active, starting, since 0.0101284s), standbys: x 2024-09-10T18:22:19.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:19 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:22:19.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:19 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:22:19.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:19 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:22:19.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:19 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:22:19.807 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:19 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:22:19.807 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:19 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:22:19.807 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:19 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:22:19.807 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:19 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:22:19.807 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:19 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:22:19.807 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:19 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:22:19.807 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:19 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:22:19.807 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:19 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:22:19.807 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:19 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:22:19.807 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:19 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:22:19.807 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:19 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:22:19.807 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:19 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:22:19.807 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:19 smithi150 ceph-mon[32761]: Manager daemon y is now available 2024-09-10T18:22:19.808 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:19 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:22:19.808 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:19 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:19.808 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:19 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:22:19.808 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:19 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:22:19.808 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:19 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:22:19.808 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:19 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:22:19.808 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:19 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:22:20.305 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:22:20 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:22:20.342 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:20 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:22:20.192+0000 7fae65488640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:18:22:20] ENGINE Error in HTTPServer.serve 2024-09-10T18:22:20.342 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:20 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: Traceback (most recent call last): 2024-09-10T18:22:20.342 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:20 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:22:20.342 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:20 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: self._connections.run(self.expiration_interval) 2024-09-10T18:22:20.342 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:20 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:22:20.342 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:20 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: self._run(expiration_interval) 2024-09-10T18:22:20.342 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:20 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:22:20.342 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:20 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:22:20.343 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:20 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:22:20.343 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:20 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:22:20.343 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:20 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:22:20.343 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:20 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: s = self.context.wrap_socket( 2024-09-10T18:22:20.343 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:20 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:22:20.343 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:20 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: return self.sslsocket_class._create( 2024-09-10T18:22:20.343 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:20 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:22:20.343 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:20 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: self.do_handshake() 2024-09-10T18:22:20.343 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:20 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:22:20.343 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:20 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: self._sslobj.do_handshake() 2024-09-10T18:22:20.343 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:20 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:22:20.344 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:22:20 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:22:21.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:21 smithi150 ceph-mon[32761]: mgrmap e19: y(active, since 1.01143s), standbys: x 2024-09-10T18:22:21.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:21 smithi150 ceph-mon[32761]: [10/Sep/2024:18:22:20] ENGINE Bus STARTING 2024-09-10T18:22:21.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:21 smithi150 ceph-mon[32761]: [10/Sep/2024:18:22:20] ENGINE Serving on https://172.21.15.115:7150 2024-09-10T18:22:21.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:21 smithi150 ceph-mon[32761]: [10/Sep/2024:18:22:20] ENGINE Bus STARTED 2024-09-10T18:22:21.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:21 smithi150 ceph-mon[32761]: [10/Sep/2024:18:22:20] ENGINE Error in HTTPServer.serve 2024-09-10T18:22:21.305 INFO:journalctl@ceph.mon.b.smithi150.stdout: Traceback (most recent call last): 2024-09-10T18:22:21.305 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:22:21.306 INFO:journalctl@ceph.mon.b.smithi150.stdout: self._connections.run(self.expiration_interval) 2024-09-10T18:22:21.306 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:22:21.306 INFO:journalctl@ceph.mon.b.smithi150.stdout: self._run(expiration_interval) 2024-09-10T18:22:21.306 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:22:21.306 INFO:journalctl@ceph.mon.b.smithi150.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:22:21.306 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:22:21.306 INFO:journalctl@ceph.mon.b.smithi150.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:22:21.306 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:22:21.306 INFO:journalctl@ceph.mon.b.smithi150.stdout: s = self.context.wrap_socket( 2024-09-10T18:22:21.306 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:22:21.306 INFO:journalctl@ceph.mon.b.smithi150.stdout: return self.sslsocket_class._create( 2024-09-10T18:22:21.306 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:22:21.306 INFO:journalctl@ceph.mon.b.smithi150.stdout: self.do_handshake() 2024-09-10T18:22:21.306 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:22:21.307 INFO:journalctl@ceph.mon.b.smithi150.stdout: self._sslobj.do_handshake() 2024-09-10T18:22:21.307 INFO:journalctl@ceph.mon.b.smithi150.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:22:21.307 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:21 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:21.312 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:21 smithi115 ceph-mon[41073]: mgrmap e19: y(active, since 1.01143s), standbys: x 2024-09-10T18:22:21.312 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:21 smithi115 ceph-mon[41073]: [10/Sep/2024:18:22:20] ENGINE Bus STARTING 2024-09-10T18:22:21.312 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:21 smithi115 ceph-mon[41073]: [10/Sep/2024:18:22:20] ENGINE Serving on https://172.21.15.115:7150 2024-09-10T18:22:21.312 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:21 smithi115 ceph-mon[41073]: [10/Sep/2024:18:22:20] ENGINE Bus STARTED 2024-09-10T18:22:21.312 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:21 smithi115 ceph-mon[41073]: [10/Sep/2024:18:22:20] ENGINE Error in HTTPServer.serve 2024-09-10T18:22:21.312 INFO:journalctl@ceph.mon.c.smithi115.stdout: Traceback (most recent call last): 2024-09-10T18:22:21.313 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:22:21.313 INFO:journalctl@ceph.mon.c.smithi115.stdout: self._connections.run(self.expiration_interval) 2024-09-10T18:22:21.313 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:22:21.313 INFO:journalctl@ceph.mon.c.smithi115.stdout: self._run(expiration_interval) 2024-09-10T18:22:21.313 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:22:21.313 INFO:journalctl@ceph.mon.c.smithi115.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:22:21.313 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:22:21.313 INFO:journalctl@ceph.mon.c.smithi115.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:22:21.313 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:22:21.313 INFO:journalctl@ceph.mon.c.smithi115.stdout: s = self.context.wrap_socket( 2024-09-10T18:22:21.313 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:22:21.313 INFO:journalctl@ceph.mon.c.smithi115.stdout: return self.sslsocket_class._create( 2024-09-10T18:22:21.313 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:22:21.313 INFO:journalctl@ceph.mon.c.smithi115.stdout: self.do_handshake() 2024-09-10T18:22:21.314 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:22:21.314 INFO:journalctl@ceph.mon.c.smithi115.stdout: self._sslobj.do_handshake() 2024-09-10T18:22:21.314 INFO:journalctl@ceph.mon.c.smithi115.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:22:21.314 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:21 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:21.314 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:21 smithi115 ceph-mon[29603]: mgrmap e19: y(active, since 1.01143s), standbys: x 2024-09-10T18:22:21.314 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:21 smithi115 ceph-mon[29603]: [10/Sep/2024:18:22:20] ENGINE Bus STARTING 2024-09-10T18:22:21.314 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:21 smithi115 ceph-mon[29603]: [10/Sep/2024:18:22:20] ENGINE Serving on https://172.21.15.115:7150 2024-09-10T18:22:21.314 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:21 smithi115 ceph-mon[29603]: [10/Sep/2024:18:22:20] ENGINE Bus STARTED 2024-09-10T18:22:21.314 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:21 smithi115 ceph-mon[29603]: [10/Sep/2024:18:22:20] ENGINE Error in HTTPServer.serve 2024-09-10T18:22:21.314 INFO:journalctl@ceph.mon.a.smithi115.stdout: Traceback (most recent call last): 2024-09-10T18:22:21.314 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:22:21.315 INFO:journalctl@ceph.mon.a.smithi115.stdout: self._connections.run(self.expiration_interval) 2024-09-10T18:22:21.315 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:22:21.315 INFO:journalctl@ceph.mon.a.smithi115.stdout: self._run(expiration_interval) 2024-09-10T18:22:21.315 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:22:21.315 INFO:journalctl@ceph.mon.a.smithi115.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:22:21.315 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:22:21.315 INFO:journalctl@ceph.mon.a.smithi115.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:22:21.315 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:22:21.315 INFO:journalctl@ceph.mon.a.smithi115.stdout: s = self.context.wrap_socket( 2024-09-10T18:22:21.315 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:22:21.315 INFO:journalctl@ceph.mon.a.smithi115.stdout: return self.sslsocket_class._create( 2024-09-10T18:22:21.315 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:22:21.315 INFO:journalctl@ceph.mon.a.smithi115.stdout: self.do_handshake() 2024-09-10T18:22:21.315 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:22:21.316 INFO:journalctl@ceph.mon.a.smithi115.stdout: self._sslobj.do_handshake() 2024-09-10T18:22:21.316 INFO:journalctl@ceph.mon.a.smithi115.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:22:21.316 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:21 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:21.428 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:22:21.428 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":61,"fsid":"a600e180-6fa0-11ef-bcea-c7b262605968","created":"2024-09-10T18:16:27.008296+0000","modified":"2024-09-10T18:22:19.018178+0000","last_up_change":"2024-09-10T18:21:29.447953+0000","last_in_change":"2024-09-10T18:21:12.478592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T18:19:32.280261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-10T18:21:40.912985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-10T18:21:41.147358+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":56,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-10T18:21:42.479868+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-10T18:21:44.452268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-10T18:21:46.516202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"107d9598-28cd-4260-9f5d-a3b7886845c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6801","nonce":4123495067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":4123495067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":4123495067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6803","nonce":4123495067}]},"public_addr":"172.21.15.115:6801/4123495067","cluster_addr":"172.21.15.115:6802/4123495067","heartbeat_back_addr":"172.21.15.115:6804/4123495067","heartbeat_front_addr":"172.21.15.115:6803/4123495067","state":["exists","up"]},{"osd":1,"uuid":"c3cfd69f-7300-4b6e-a57b-33eefed950b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6805","nonce":2506968391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2506968391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2506968391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6807","nonce":2506968391}]},"public_addr":"172.21.15.115:6805/2506968391","cluster_addr":"172.21.15.115:6806/2506968391","heartbeat_back_addr":"172.21.15.115:6808/2506968391","heartbeat_front_addr":"172.21.15.115:6807/2506968391","state":["exists","up"]},{"osd":2,"uuid":"9abe6878-817d-4c81-b4b8-db61147c2f94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6809","nonce":343149052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":343149052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":343149052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6811","nonce":343149052}]},"public_addr":"172.21.15.115:6809/343149052","cluster_addr":"172.21.15.115:6810/343149052","heartbeat_back_addr":"172.21.15.115:6812/343149052","heartbeat_front_addr":"172.21.15.115:6811/343149052","state":["exists","up"]},{"osd":3,"uuid":"4fefb92a-fc8d-4a28-9c89-1b7e214c8e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6813","nonce":3162056893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":3162056893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3162056893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":3162056893}]},"public_addr":"172.21.15.115:6813/3162056893","cluster_addr":"172.21.15.115:6814/3162056893","heartbeat_back_addr":"172.21.15.115:6816/3162056893","heartbeat_front_addr":"172.21.15.115:6815/3162056893","state":["exists","up"]},{"osd":4,"uuid":"fa14f44d-1b45-490d-a686-77da88336564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1929277477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6801","nonce":1929277477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6803","nonce":1929277477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1929277477}]},"public_addr":"172.21.15.150:6800/1929277477","cluster_addr":"172.21.15.150:6801/1929277477","heartbeat_back_addr":"172.21.15.150:6803/1929277477","heartbeat_front_addr":"172.21.15.150:6802/1929277477","state":["exists","up"]},{"osd":5,"uuid":"40cabcd5-6874-45d2-bfaf-d737f834e7f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3667323015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6805","nonce":3667323015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6807","nonce":3667323015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3667323015}]},"public_addr":"172.21.15.150:6804/3667323015","cluster_addr":"172.21.15.150:6805/3667323015","heartbeat_back_addr":"172.21.15.150:6807/3667323015","heartbeat_front_addr":"172.21.15.150:6806/3667323015","state":["exists","up"]},{"osd":6,"uuid":"d871feca-36dd-4180-becc-e3e0dd9b4319","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3335870479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3335870479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6811","nonce":3335870479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3335870479}]},"public_addr":"172.21.15.150:6808/3335870479","cluster_addr":"172.21.15.150:6809/3335870479","heartbeat_back_addr":"172.21.15.150:6811/3335870479","heartbeat_front_addr":"172.21.15.150:6810/3335870479","state":["exists","up"]},{"osd":7,"uuid":"0aaa1287-7337-49a5-a832-709aa3980834","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":672367088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":672367088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6815","nonce":672367088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":672367088}]},"public_addr":"172.21.15.150:6812/672367088","cluster_addr":"172.21.15.150:6813/672367088","heartbeat_back_addr":"172.21.15.150:6815/672367088","heartbeat_front_addr":"172.21.15.150:6814/672367088","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:18:41.264707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:19:05.260563+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:19:29.810350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:19:54.613874+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:20:16.294916+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:20:40.099086+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:21:03.703231+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:21:27.515712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/936574235":"2024-09-11T18:16:59.107583+0000","172.21.15.115:0/337878570":"2024-09-11T18:22:19.018149+0000","172.21.15.115:0/3979277447":"2024-09-11T18:16:59.107583+0000","172.21.15.115:0/1020299194":"2024-09-11T18:16:59.107583+0000","172.21.15.115:0/553409925":"2024-09-11T18:17:21.977918+0000","172.21.15.115:6800/2658904986":"2024-09-11T18:16:59.107583+0000","172.21.15.115:6800/3682910233":"2024-09-11T18:17:21.977918+0000","172.21.15.115:0/4136881386":"2024-09-11T18:22:19.018149+0000","172.21.15.115:0/3905115041":"2024-09-11T18:17:21.977918+0000","172.21.15.115:0/2382563730":"2024-09-11T18:17:21.977918+0000","172.21.15.115:0/570209858":"2024-09-11T18:22:19.018149+0000","172.21.15.115:0/171922759":"2024-09-11T18:22:19.018149+0000","172.21.15.115:6800/667858637":"2024-09-11T18:22:19.018149+0000","172.21.15.115:0/498921679":"2024-09-11T18:22:19.018149+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T18:22:22.202 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:22 smithi115 ceph-mon[29603]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:22:22.202 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:22 smithi115 ceph-mon[29603]: mgrmap e20: y(active, since 2s), standbys: x 2024-09-10T18:22:22.202 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:22 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1026407362' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:22:22.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:22 smithi150 ceph-mon[32761]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:22:22.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:22 smithi150 ceph-mon[32761]: mgrmap e20: y(active, since 2s), standbys: x 2024-09-10T18:22:22.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:22 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1026407362' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:22:22.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:22 smithi115 ceph-mon[41073]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:22:22.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:22 smithi115 ceph-mon[41073]: mgrmap e20: y(active, since 2s), standbys: x 2024-09-10T18:22:22.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:22 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1026407362' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:22:22.601 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-10T18:22:22.694 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-10T18:22:22.695 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-10T18:22:22.695 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-10T18:22:22.695 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-10T18:22:22.696 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-10T18:22:22.696 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-10T18:22:22.697 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-10T18:22:23.172 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:22:23.206 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:22:23.232 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:22:23.251 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:22:23.480 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:23 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:23.480 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:23 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:23.480 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:23 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:23.481 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:23 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:23.481 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:23 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:23.481 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:23 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:23.481 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:23 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:23.481 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:23 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:23.481 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:23 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:23.481 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:23 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:23.481 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:23 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:23.481 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:23 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:23.481 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:23 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:23.481 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:23 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:23.482 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:23 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:23.482 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:23 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:23.482 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:23 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:23.482 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:23 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:23.482 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:23 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:23.482 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:23 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:23.482 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:23 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:23.482 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:23 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:23.482 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:23 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:23.482 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:23 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:23.482 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:23 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:23.482 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:23 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:23.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:23 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:23.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:23 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:23.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:23 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:23.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:23 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:23.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:23 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:23.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:23 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:23.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:23 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:23.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:23 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:23.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:23 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:23.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:23 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:23.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:23 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:23.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:23 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:23.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:23 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:24.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:24 smithi115 ceph-mon[41073]: Adjusting osd_memory_target on smithi150 to 3760M 2024-09-10T18:22:24.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:24 smithi115 ceph-mon[41073]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:22:24.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:24 smithi115 ceph-mon[41073]: mgrmap e21: y(active, since 4s), standbys: x 2024-09-10T18:22:24.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:24 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:24.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:24 smithi115 ceph-mon[29603]: Adjusting osd_memory_target on smithi150 to 3760M 2024-09-10T18:22:24.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:24 smithi115 ceph-mon[29603]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:22:24.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:24 smithi115 ceph-mon[29603]: mgrmap e21: y(active, since 4s), standbys: x 2024-09-10T18:22:24.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:24 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:24.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:24 smithi150 ceph-mon[32761]: Adjusting osd_memory_target on smithi150 to 3760M 2024-09-10T18:22:24.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:24 smithi150 ceph-mon[32761]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:22:24.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:24 smithi150 ceph-mon[32761]: mgrmap e21: y(active, since 4s), standbys: x 2024-09-10T18:22:24.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:24 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:25.319 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:22:25.320 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:22:25.320 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:22:25.325 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:22:25.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:25 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:25.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:25 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:25.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:25 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:26.346 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:26 smithi115 ceph-mon[41073]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-10T18:22:26.346 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:26 smithi115 ceph-mon[29603]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-10T18:22:26.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:26 smithi150 ceph-mon[32761]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-10T18:22:28.422 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:28 smithi115 ceph-mon[41073]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-10T18:22:28.423 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:28 smithi115 ceph-mon[29603]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-10T18:22:28.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:28 smithi150 ceph-mon[32761]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-10T18:22:29.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:28 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:22:28] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:22:30.290 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[29603]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T18:22:30.290 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:30.290 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:30.290 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:30.290 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:30.290 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:30.290 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:30.290 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:30.291 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:30.291 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:30.291 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:30.291 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[29603]: Adjusting osd_memory_target on smithi115 to 3760M 2024-09-10T18:22:30.291 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:30.291 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:22:30.291 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:22:30.291 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[29603]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-10T18:22:30.291 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[29603]: Updating smithi150:/etc/ceph/ceph.conf 2024-09-10T18:22:30.291 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[29603]: Updating smithi115:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:22:30.291 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[29603]: Updating smithi150:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:22:30.291 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:30.291 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:30.292 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:30.292 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:30.292 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:30.292 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[41073]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T18:22:30.292 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:30.292 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:30.292 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:30.292 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:30.292 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:30.292 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:30.292 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:30.293 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:30.293 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:30.293 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:30.293 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[41073]: Adjusting osd_memory_target on smithi115 to 3760M 2024-09-10T18:22:30.293 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:30.293 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:22:30.293 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:22:30.293 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[41073]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-10T18:22:30.293 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[41073]: Updating smithi150:/etc/ceph/ceph.conf 2024-09-10T18:22:30.293 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[41073]: Updating smithi115:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:22:30.293 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[41073]: Updating smithi150:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:22:30.293 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:30.293 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:30.293 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:30.294 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:30.294 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:30 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:30.305 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:22:30 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:22:30.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:30 smithi150 ceph-mon[32761]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T18:22:30.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:30 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:30.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:30 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:30.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:30 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:30.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:30 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:30.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:30 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:30.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:30 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:30.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:30 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:30.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:30 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:30.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:30 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:30.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:30 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:22:30.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:30 smithi150 ceph-mon[32761]: Adjusting osd_memory_target on smithi115 to 3760M 2024-09-10T18:22:30.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:30 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:30.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:30 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:22:30.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:30 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:22:30.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:30 smithi150 ceph-mon[32761]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-10T18:22:30.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:30 smithi150 ceph-mon[32761]: Updating smithi150:/etc/ceph/ceph.conf 2024-09-10T18:22:30.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:30 smithi150 ceph-mon[32761]: Updating smithi115:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:22:30.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:30 smithi150 ceph-mon[32761]: Updating smithi150:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:22:30.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:30 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:30.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:30 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:30.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:30 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:30.307 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:30 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:30.307 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:30 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:22:31.463 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:31 smithi115 ceph-mon[41073]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:22:31.464 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:31 smithi115 ceph-mon[41073]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:22:31.464 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:31 smithi115 ceph-mon[41073]: Updating smithi115:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:22:31.464 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:31 smithi115 ceph-mon[41073]: Updating smithi150:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:22:31.464 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:31 smithi115 ceph-mon[41073]: Deploying daemon alertmanager.a on smithi115 2024-09-10T18:22:31.464 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:31 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:22:31.464 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:31 smithi115 ceph-mon[29603]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:22:31.464 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:31 smithi115 ceph-mon[29603]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:22:31.464 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:31 smithi115 ceph-mon[29603]: Updating smithi115:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:22:31.464 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:31 smithi115 ceph-mon[29603]: Updating smithi150:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:22:31.464 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:31 smithi115 ceph-mon[29603]: Deploying daemon alertmanager.a on smithi115 2024-09-10T18:22:31.465 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:31 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:22:31.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:31 smithi150 ceph-mon[32761]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:22:31.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:31 smithi150 ceph-mon[32761]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:22:31.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:31 smithi150 ceph-mon[32761]: Updating smithi115:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:22:31.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:31 smithi150 ceph-mon[32761]: Updating smithi150:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:22:31.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:31 smithi150 ceph-mon[32761]: Deploying daemon alertmanager.a on smithi115 2024-09-10T18:22:31.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:31 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:22:32.442 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:32 smithi150 ceph-mon[32761]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:22:32.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:32 smithi115 ceph-mon[41073]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:22:32.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:32 smithi115 ceph-mon[29603]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:22:34.265 INFO:teuthology.orchestra.run.smithi115.stdout:55834574891 2024-09-10T18:22:34.266 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-10T18:22:34.359 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:34 smithi115 ceph-mon[41073]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T18:22:34.359 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:34 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:22:34.359 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:34 smithi115 ceph-mon[29603]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T18:22:34.359 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:34 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:22:34.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:34 smithi150 ceph-mon[32761]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T18:22:34.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:34 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:22:35.740 INFO:teuthology.orchestra.run.smithi115.stdout:154618822680 2024-09-10T18:22:35.741 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-10T18:22:36.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:36 smithi115 ceph-mon[41073]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:22:36.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:36 smithi115 ceph-mon[29603]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:22:36.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:36 smithi150 ceph-mon[32761]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:22:36.714 INFO:teuthology.orchestra.run.smithi115.stdout:180388626452 2024-09-10T18:22:36.714 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-10T18:22:37.293 INFO:teuthology.orchestra.run.smithi115.stdout:34359738417 2024-09-10T18:22:37.293 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-10T18:22:38.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:38 smithi115 ceph-mon[41073]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:22:38.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:38 smithi115 ceph-mon[29603]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:22:38.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:38 smithi150 ceph-mon[32761]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:22:39.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:38 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:22:38] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:22:39.898 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:22:40.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:40 smithi115 ceph-mon[41073]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:22:40.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:40 smithi115 ceph-mon[29603]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:22:40.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:40 smithi150 ceph-mon[32761]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:22:40.555 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:22:40 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:22:41.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:41 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:22:41.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:41 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:22:41.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:41 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:22:41.673 INFO:teuthology.orchestra.run.smithi115.stdout:77309411368 2024-09-10T18:22:41.674 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-10T18:22:42.173 INFO:teuthology.orchestra.run.smithi115.stdout:206158430225 2024-09-10T18:22:42.174 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-10T18:22:42.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:42 smithi150 ceph-mon[32761]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:42.406 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:42 smithi115 ceph-mon[41073]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:42.406 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:42 smithi115 ceph-mon[29603]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:42.853 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:22:43.323 INFO:teuthology.orchestra.run.smithi115.stdout:128849018910 2024-09-10T18:22:43.323 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-10T18:22:44.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:44 smithi115 ceph-mon[29603]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:22:44.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:44 smithi115 ceph-mon[41073]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:22:44.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:44 smithi150 ceph-mon[32761]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:22:46.054 INFO:teuthology.orchestra.run.smithi115.stdout:107374182435 2024-09-10T18:22:46.054 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-10T18:22:46.385 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:46 smithi115 ceph-mon[41073]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:46.385 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:46 smithi115 ceph-mon[29603]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:46.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:46 smithi150 ceph-mon[32761]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:46.632 INFO:teuthology.orchestra.run.smithi115.stdout:55834574894 2024-09-10T18:22:47.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:47 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1932675246' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T18:22:47.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:47 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1932675246' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T18:22:47.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:47 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1932675246' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T18:22:47.615 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574891 got 55834574894 for osd.1 2024-09-10T18:22:47.615 DEBUG:teuthology.parallel:result is None 2024-09-10T18:22:47.648 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:22:47.648 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:22:47.649 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:22:47.652 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:22:48.224 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:22:48.316 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:48 smithi115 ceph-mon[41073]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:22:48.316 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:48 smithi115 ceph-mon[29603]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:22:48.554 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:48 smithi150 ceph-mon[32761]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:22:48.789 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:22:49.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:48 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:22:48] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:22:49.467 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:49 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:22:49.468 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:49 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:22:49.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:49 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:22:50.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:50 smithi115 ceph-mon[41073]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:22:50.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:50 smithi115 ceph-mon[29603]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:22:50.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:50 smithi150 ceph-mon[32761]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:22:50.555 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:22:50 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:22:51.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:51 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:22:51.416 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:51 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:22:51.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:51 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:22:51.929 INFO:teuthology.orchestra.run.smithi115.stdout:154618822684 2024-09-10T18:22:52.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:52 smithi115 ceph-mon[41073]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:52.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:52 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/3298481142' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T18:22:52.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:52 smithi115 ceph-mon[29603]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:52.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:52 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/3298481142' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T18:22:52.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:52 smithi150 ceph-mon[32761]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:52.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:52 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/3298481142' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T18:22:53.495 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822680 got 154618822684 for osd.5 2024-09-10T18:22:53.495 DEBUG:teuthology.parallel:result is None 2024-09-10T18:22:54.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:54 smithi115 ceph-mon[41073]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:22:54.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:54 smithi115 ceph-mon[29603]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:22:54.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:54 smithi150 ceph-mon[32761]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:22:56.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:56 smithi115 ceph-mon[41073]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:56.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:56 smithi115 ceph-mon[29603]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:56.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:56 smithi150 ceph-mon[32761]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:58.395 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:58 smithi115 ceph-mon[41073]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:22:58.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:58 smithi115 ceph-mon[29603]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:22:58.427 INFO:teuthology.orchestra.run.smithi115.stdout:107374182439 2024-09-10T18:22:58.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:58 smithi150 ceph-mon[32761]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:22:58.645 INFO:teuthology.orchestra.run.smithi115.stdout:206158430228 2024-09-10T18:22:58.857 INFO:teuthology.orchestra.run.smithi115.stdout:180388626456 2024-09-10T18:22:59.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:22:58 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:22:58] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:22:59.266 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430225 got 206158430228 for osd.7 2024-09-10T18:22:59.266 DEBUG:teuthology.parallel:result is None 2024-09-10T18:22:59.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:59 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/878538838' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T18:22:59.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:59 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/2188341816' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T18:22:59.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:22:59 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1164887785' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T18:22:59.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:59 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/878538838' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T18:22:59.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:59 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/2188341816' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T18:22:59.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:22:59 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1164887785' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T18:22:59.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:59 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/878538838' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T18:22:59.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:59 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/2188341816' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T18:22:59.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:22:59 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1164887785' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T18:22:59.852 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182435 got 107374182439 for osd.3 2024-09-10T18:22:59.852 DEBUG:teuthology.parallel:result is None 2024-09-10T18:23:00.451 INFO:teuthology.orchestra.run.smithi115.stdout:77309411372 2024-09-10T18:23:00.451 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:00 smithi115 ceph-mon[29603]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:00.451 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:00 smithi115 ceph-mon[41073]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:00.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:00 smithi150 ceph-mon[32761]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:00.555 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:23:00 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:23:01.139 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:00 smithi115 podman[76584]: 2024-09-10 18:23:00.938277407 +0000 UTC m=+1.038978647 container create 7146acf5a1ddc98348449435951921525efc965715c4bb600563a024f815b0cb (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:23:01.477 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:01 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:23:01.477 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:01 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/994074114' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T18:23:01.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:01 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:23:01.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:01 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/994074114' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T18:23:01.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:01 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:23:01.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:01 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/994074114' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T18:23:01.551 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626452 got 180388626456 for osd.6 2024-09-10T18:23:01.552 DEBUG:teuthology.parallel:result is None 2024-09-10T18:23:02.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:02 smithi115 ceph-mon[41073]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:02.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:02 smithi115 ceph-mon[29603]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:02.235 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:02 smithi115 podman[76584]: 2024-09-10 18:23:02.071121089 +0000 UTC m=+2.171822327 container init 7146acf5a1ddc98348449435951921525efc965715c4bb600563a024f815b0cb (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:23:02.236 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:02 smithi115 podman[76584]: 2024-09-10 18:23:02.076726556 +0000 UTC m=+2.177427792 container start 7146acf5a1ddc98348449435951921525efc965715c4bb600563a024f815b0cb (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:23:02.236 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:02 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a[76671]: ts=2024-09-10T18:23:02.089Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-10T18:23:02.236 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:02 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a[76671]: ts=2024-09-10T18:23:02.089Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-10T18:23:02.236 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:02 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a[76671]: ts=2024-09-10T18:23:02.090Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.115 port=9094 2024-09-10T18:23:02.236 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:02 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a[76671]: ts=2024-09-10T18:23:02.091Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-10T18:23:02.236 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:02 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a[76671]: ts=2024-09-10T18:23:02.114Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-10T18:23:02.236 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:02 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a[76671]: ts=2024-09-10T18:23:02.114Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-10T18:23:02.236 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:02 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a[76671]: ts=2024-09-10T18:23:02.117Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-10T18:23:02.236 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:02 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a[76671]: ts=2024-09-10T18:23:02.117Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-10T18:23:02.236 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:02 smithi115 bash[76584]: 7146acf5a1ddc98348449435951921525efc965715c4bb600563a024f815b0cb 2024-09-10T18:23:02.236 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:02 smithi115 systemd[1]: Started Ceph alertmanager.a for a600e180-6fa0-11ef-bcea-c7b262605968. 2024-09-10T18:23:02.554 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:02 smithi150 ceph-mon[32761]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:02.583 INFO:teuthology.orchestra.run.smithi115.stdout:128849018914 2024-09-10T18:23:02.583 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:23:02 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: [10/Sep/2024:18:23:02] ENGINE Bus STOPPING 2024-09-10T18:23:02.877 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:23:02 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: [10/Sep/2024:18:23:02] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:23:02.877 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:23:02 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: [10/Sep/2024:18:23:02] ENGINE Bus STOPPED 2024-09-10T18:23:02.877 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:23:02 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: [10/Sep/2024:18:23:02] ENGINE Bus STARTING 2024-09-10T18:23:02.878 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:23:02 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: [10/Sep/2024:18:23:02] ENGINE Serving on http://:::9283 2024-09-10T18:23:02.878 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:23:02 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: [10/Sep/2024:18:23:02] ENGINE Bus STARTED 2024-09-10T18:23:02.960 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411368 got 77309411372 for osd.2 2024-09-10T18:23:02.961 DEBUG:teuthology.parallel:result is None 2024-09-10T18:23:03.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:03 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:03.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:03 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:03.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:03 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:03.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:03 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:03.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:03 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:03.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:03 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:03.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:03 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:23:03.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:03 smithi115 ceph-mon[41073]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:23:03.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:03 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:03.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:03 smithi115 ceph-mon[41073]: Deploying daemon grafana.a on smithi150 2024-09-10T18:23:03.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:03 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1822787580' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T18:23:03.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:03 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:03.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:03 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:03.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:03 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:03.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:03 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:03.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:03 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:03.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:03 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:03.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:03 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:23:03.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:03 smithi115 ceph-mon[29603]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:23:03.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:03 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:03.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:03 smithi115 ceph-mon[29603]: Deploying daemon grafana.a on smithi150 2024-09-10T18:23:03.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:03 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1822787580' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T18:23:03.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:03 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:03.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:03 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:03.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:03 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:03.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:03 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:03.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:03 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:03.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:03 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:03.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:03 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:23:03.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:03 smithi150 ceph-mon[32761]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:23:03.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:03 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:03.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:03 smithi150 ceph-mon[32761]: Deploying daemon grafana.a on smithi150 2024-09-10T18:23:03.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:03 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1822787580' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T18:23:03.559 INFO:teuthology.orchestra.run.smithi115.stdout:34359738423 2024-09-10T18:23:04.235 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:04 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a[76671]: ts=2024-09-10T18:23:04.091Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000582157s 2024-09-10T18:23:04.356 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018910 got 128849018914 for osd.4 2024-09-10T18:23:04.356 DEBUG:teuthology.parallel:result is None 2024-09-10T18:23:04.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:04 smithi115 ceph-mon[29603]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:04.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:04 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1460493154' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T18:23:04.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:04 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:23:04.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:04 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:04.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:04 smithi115 ceph-mon[41073]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:04.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:04 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1460493154' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T18:23:04.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:04 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:23:04.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:04 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:04.803 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738417 got 34359738423 for osd.0 2024-09-10T18:23:04.803 DEBUG:teuthology.parallel:result is None 2024-09-10T18:23:04.804 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T18:23:04.804 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T18:23:04.804 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:04 smithi150 ceph-mon[32761]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:04.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:04 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1460493154' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T18:23:04.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:04 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:23:04.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:04 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:05.052 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:23:06.359 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:06 smithi115 ceph-mon[41073]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:06.360 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:06 smithi115 ceph-mon[29603]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:06.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:06 smithi150 ceph-mon[32761]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:07.093 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:23:07.095 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T18:23:07.740 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-09-10T18:23:07.046822+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":929,"num_read_kb":786,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":548,"ondisk_log_size":548,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385384,"kb_used_data":6548,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575832,"statfs":{"total":767960285184,"available":765517651968,"internally_reserved":0,"allocated":6705152,"data_stored":3631893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003326"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.004738+0000","last_change":"2024-09-10T18:21:48.249635+0000","last_active":"2024-09-10T18:21:52.004738+0000","last_peered":"2024-09-10T18:21:52.004738+0000","last_clean":"2024-09-10T18:21:52.004738+0000","last_became_active":"2024-09-10T18:21:48.249190+0000","last_became_peered":"2024-09-10T18:21:48.249190+0000","last_unstale":"2024-09-10T18:21:52.004738+0000","last_undegraded":"2024-09-10T18:21:52.004738+0000","last_fullsized":"2024-09-10T18:21:52.004738+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:27:20.808410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.322935+0000","last_change":"2024-09-10T18:21:42.193527+0000","last_active":"2024-09-10T18:21:52.322935+0000","last_peered":"2024-09-10T18:21:52.322935+0000","last_clean":"2024-09-10T18:21:52.322935+0000","last_became_active":"2024-09-10T18:21:42.193245+0000","last_became_peered":"2024-09-10T18:21:42.193245+0000","last_unstale":"2024-09-10T18:21:52.322935+0000","last_undegraded":"2024-09-10T18:21:52.322935+0000","last_fullsized":"2024-09-10T18:21:52.322935+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:05:31.863462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.004796+0000","last_change":"2024-09-10T18:21:44.214718+0000","last_active":"2024-09-10T18:21:52.004796+0000","last_peered":"2024-09-10T18:21:52.004796+0000","last_clean":"2024-09-10T18:21:52.004796+0000","last_became_active":"2024-09-10T18:21:44.214441+0000","last_became_peered":"2024-09-10T18:21:44.214441+0000","last_unstale":"2024-09-10T18:21:52.004796+0000","last_undegraded":"2024-09-10T18:21:52.004796+0000","last_fullsized":"2024-09-10T18:21:52.004796+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:56.693341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.006028+0000","last_change":"2024-09-10T18:21:46.217211+0000","last_active":"2024-09-10T18:21:52.006028+0000","last_peered":"2024-09-10T18:21:52.006028+0000","last_clean":"2024-09-10T18:21:52.006028+0000","last_became_active":"2024-09-10T18:21:46.216944+0000","last_became_peered":"2024-09-10T18:21:46.216944+0000","last_unstale":"2024-09-10T18:21:52.006028+0000","last_undegraded":"2024-09-10T18:21:52.006028+0000","last_fullsized":"2024-09-10T18:21:52.006028+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:49.462516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.005614+0000","last_change":"2024-09-10T18:21:42.193333+0000","last_active":"2024-09-10T18:21:52.005614+0000","last_peered":"2024-09-10T18:21:52.005614+0000","last_clean":"2024-09-10T18:21:52.005614+0000","last_became_active":"2024-09-10T18:21:42.193080+0000","last_became_peered":"2024-09-10T18:21:42.193080+0000","last_unstale":"2024-09-10T18:21:52.005614+0000","last_undegraded":"2024-09-10T18:21:52.005614+0000","last_fullsized":"2024-09-10T18:21:52.005614+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:56:14.627342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.006366+0000","last_change":"2024-09-10T18:21:44.213781+0000","last_active":"2024-09-10T18:21:52.006366+0000","last_peered":"2024-09-10T18:21:52.006366+0000","last_clean":"2024-09-10T18:21:52.006366+0000","last_became_active":"2024-09-10T18:21:44.213506+0000","last_became_peered":"2024-09-10T18:21:44.213506+0000","last_unstale":"2024-09-10T18:21:52.006366+0000","last_undegraded":"2024-09-10T18:21:52.006366+0000","last_fullsized":"2024-09-10T18:21:52.006366+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:55:23.286343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:53.103332+0000","last_change":"2024-09-10T18:21:46.211107+0000","last_active":"2024-09-10T18:21:53.103332+0000","last_peered":"2024-09-10T18:21:53.103332+0000","last_clean":"2024-09-10T18:21:53.103332+0000","last_became_active":"2024-09-10T18:21:46.210710+0000","last_became_peered":"2024-09-10T18:21:46.210710+0000","last_unstale":"2024-09-10T18:21:53.103332+0000","last_undegraded":"2024-09-10T18:21:53.103332+0000","last_fullsized":"2024-09-10T18:21:53.103332+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:38:58.779887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.006059+0000","last_change":"2024-09-10T18:21:48.215402+0000","last_active":"2024-09-10T18:21:52.006059+0000","last_peered":"2024-09-10T18:21:52.006059+0000","last_clean":"2024-09-10T18:21:52.006059+0000","last_became_active":"2024-09-10T18:21:48.215147+0000","last_became_peered":"2024-09-10T18:21:48.215147+0000","last_unstale":"2024-09-10T18:21:52.006059+0000","last_undegraded":"2024-09-10T18:21:52.006059+0000","last_fullsized":"2024-09-10T18:21:52.006059+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:51.469720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"52'1","reported_seq":27,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.397414+0000","last_change":"2024-09-10T18:21:42.196527+0000","last_active":"2024-09-10T18:21:52.397414+0000","last_peered":"2024-09-10T18:21:52.397414+0000","last_clean":"2024-09-10T18:21:52.397414+0000","last_became_active":"2024-09-10T18:21:42.196243+0000","last_became_peered":"2024-09-10T18:21:42.196243+0000","last_unstale":"2024-09-10T18:21:52.397414+0000","last_undegraded":"2024-09-10T18:21:52.397414+0000","last_fullsized":"2024-09-10T18:21:52.397414+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:28.682300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"59'5","reported_seq":26,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.006227+0000","last_change":"2024-09-10T18:21:44.218505+0000","last_active":"2024-09-10T18:21:52.006227+0000","last_peered":"2024-09-10T18:21:52.006227+0000","last_clean":"2024-09-10T18:21:52.006227+0000","last_became_active":"2024-09-10T18:21:44.218210+0000","last_became_peered":"2024-09-10T18:21:44.218210+0000","last_unstale":"2024-09-10T18:21:52.006227+0000","last_undegraded":"2024-09-10T18:21:52.006227+0000","last_fullsized":"2024-09-10T18:21:52.006227+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:16.477789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.396922+0000","last_change":"2024-09-10T18:21:46.216369+0000","last_active":"2024-09-10T18:21:52.396922+0000","last_peered":"2024-09-10T18:21:52.396922+0000","last_clean":"2024-09-10T18:21:52.396922+0000","last_became_active":"2024-09-10T18:21:46.216087+0000","last_became_peered":"2024-09-10T18:21:46.216087+0000","last_unstale":"2024-09-10T18:21:52.396922+0000","last_undegraded":"2024-09-10T18:21:52.396922+0000","last_fullsized":"2024-09-10T18:21:52.396922+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:40.439486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.382555+0000","last_change":"2024-09-10T18:21:48.220939+0000","last_active":"2024-09-10T18:21:52.382555+0000","last_peered":"2024-09-10T18:21:52.382555+0000","last_clean":"2024-09-10T18:21:52.382555+0000","last_became_active":"2024-09-10T18:21:48.220645+0000","last_became_peered":"2024-09-10T18:21:48.220645+0000","last_unstale":"2024-09-10T18:21:52.382555+0000","last_undegraded":"2024-09-10T18:21:52.382555+0000","last_fullsized":"2024-09-10T18:21:52.382555+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:58.021282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.397263+0000","last_change":"2024-09-10T18:21:42.192801+0000","last_active":"2024-09-10T18:21:52.397263+0000","last_peered":"2024-09-10T18:21:52.397263+0000","last_clean":"2024-09-10T18:21:52.397263+0000","last_became_active":"2024-09-10T18:21:42.192299+0000","last_became_peered":"2024-09-10T18:21:42.192299+0000","last_unstale":"2024-09-10T18:21:52.397263+0000","last_undegraded":"2024-09-10T18:21:52.397263+0000","last_fullsized":"2024-09-10T18:21:52.397263+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:29:09.257584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.006281+0000","last_change":"2024-09-10T18:21:44.212626+0000","last_active":"2024-09-10T18:21:52.006281+0000","last_peered":"2024-09-10T18:21:52.006281+0000","last_clean":"2024-09-10T18:21:52.006281+0000","last_became_active":"2024-09-10T18:21:44.211888+0000","last_became_peered":"2024-09-10T18:21:44.211888+0000","last_unstale":"2024-09-10T18:21:52.006281+0000","last_undegraded":"2024-09-10T18:21:52.006281+0000","last_fullsized":"2024-09-10T18:21:52.006281+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:56.119082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.382747+0000","last_change":"2024-09-10T18:21:46.223065+0000","last_active":"2024-09-10T18:21:52.382747+0000","last_peered":"2024-09-10T18:21:52.382747+0000","last_clean":"2024-09-10T18:21:52.382747+0000","last_became_active":"2024-09-10T18:21:46.222792+0000","last_became_peered":"2024-09-10T18:21:46.222792+0000","last_unstale":"2024-09-10T18:21:52.382747+0000","last_undegraded":"2024-09-10T18:21:52.382747+0000","last_fullsized":"2024-09-10T18:21:52.382747+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:47.666989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.322005+0000","last_change":"2024-09-10T18:21:48.224521+0000","last_active":"2024-09-10T18:21:52.322005+0000","last_peered":"2024-09-10T18:21:52.322005+0000","last_clean":"2024-09-10T18:21:52.322005+0000","last_became_active":"2024-09-10T18:21:48.224241+0000","last_became_peered":"2024-09-10T18:21:48.224241+0000","last_unstale":"2024-09-10T18:21:52.322005+0000","last_undegraded":"2024-09-10T18:21:52.322005+0000","last_fullsized":"2024-09-10T18:21:52.322005+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:45:52.473703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"59'17","reported_seq":44,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.006688+0000","last_change":"2024-09-10T18:21:44.213864+0000","last_active":"2024-09-10T18:21:52.006688+0000","last_peered":"2024-09-10T18:21:52.006688+0000","last_clean":"2024-09-10T18:21:52.006688+0000","last_became_active":"2024-09-10T18:21:44.213420+0000","last_became_peered":"2024-09-10T18:21:44.213420+0000","last_unstale":"2024-09-10T18:21:52.006688+0000","last_undegraded":"2024-09-10T18:21:52.006688+0000","last_fullsized":"2024-09-10T18:21:52.006688+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:39.523394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.397448+0000","last_change":"2024-09-10T18:21:42.191952+0000","last_active":"2024-09-10T18:21:52.397448+0000","last_peered":"2024-09-10T18:21:52.397448+0000","last_clean":"2024-09-10T18:21:52.397448+0000","last_became_active":"2024-09-10T18:21:42.191546+0000","last_became_peered":"2024-09-10T18:21:42.191546+0000","last_unstale":"2024-09-10T18:21:52.397448+0000","last_undegraded":"2024-09-10T18:21:52.397448+0000","last_fullsized":"2024-09-10T18:21:52.397448+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:40.565015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:53.103985+0000","last_change":"2024-09-10T18:21:46.213690+0000","last_active":"2024-09-10T18:21:53.103985+0000","last_peered":"2024-09-10T18:21:53.103985+0000","last_clean":"2024-09-10T18:21:53.103985+0000","last_became_active":"2024-09-10T18:21:46.213385+0000","last_became_peered":"2024-09-10T18:21:46.213385+0000","last_unstale":"2024-09-10T18:21:53.103985+0000","last_undegraded":"2024-09-10T18:21:53.103985+0000","last_fullsized":"2024-09-10T18:21:53.103985+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:36:09.241715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.004382+0000","last_change":"2024-09-10T18:21:48.222236+0000","last_active":"2024-09-10T18:21:52.004382+0000","last_peered":"2024-09-10T18:21:52.004382+0000","last_clean":"2024-09-10T18:21:52.004382+0000","last_became_active":"2024-09-10T18:21:48.221959+0000","last_became_peered":"2024-09-10T18:21:48.221959+0000","last_unstale":"2024-09-10T18:21:52.004382+0000","last_undegraded":"2024-09-10T18:21:52.004382+0000","last_fullsized":"2024-09-10T18:21:52.004382+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:58.843453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.006528+0000","last_change":"2024-09-10T18:21:44.212233+0000","last_active":"2024-09-10T18:21:52.006528+0000","last_peered":"2024-09-10T18:21:52.006528+0000","last_clean":"2024-09-10T18:21:52.006528+0000","last_became_active":"2024-09-10T18:21:44.211440+0000","last_became_peered":"2024-09-10T18:21:44.211440+0000","last_unstale":"2024-09-10T18:21:52.006528+0000","last_undegraded":"2024-09-10T18:21:52.006528+0000","last_fullsized":"2024-09-10T18:21:52.006528+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:32.793112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:53.103764+0000","last_change":"2024-09-10T18:21:42.189586+0000","last_active":"2024-09-10T18:21:53.103764+0000","last_peered":"2024-09-10T18:21:53.103764+0000","last_clean":"2024-09-10T18:21:53.103764+0000","last_became_active":"2024-09-10T18:21:42.189177+0000","last_became_peered":"2024-09-10T18:21:42.189177+0000","last_unstale":"2024-09-10T18:21:53.103764+0000","last_undegraded":"2024-09-10T18:21:53.103764+0000","last_fullsized":"2024-09-10T18:21:53.103764+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:16:41.693895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"59'8","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:22:55.686544+0000","last_change":"2024-09-10T18:21:46.217678+0000","last_active":"2024-09-10T18:22:55.686544+0000","last_peered":"2024-09-10T18:22:55.686544+0000","last_clean":"2024-09-10T18:22:55.686544+0000","last_became_active":"2024-09-10T18:21:46.217352+0000","last_became_peered":"2024-09-10T18:21:46.217352+0000","last_unstale":"2024-09-10T18:22:55.686544+0000","last_undegraded":"2024-09-10T18:22:55.686544+0000","last_fullsized":"2024-09-10T18:22:55.686544+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:10:35.296112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.006522+0000","last_change":"2024-09-10T18:21:48.217792+0000","last_active":"2024-09-10T18:21:52.006522+0000","last_peered":"2024-09-10T18:21:52.006522+0000","last_clean":"2024-09-10T18:21:52.006522+0000","last_became_active":"2024-09-10T18:21:48.217514+0000","last_became_peered":"2024-09-10T18:21:48.217514+0000","last_unstale":"2024-09-10T18:21:52.006522+0000","last_undegraded":"2024-09-10T18:21:52.006522+0000","last_fullsized":"2024-09-10T18:21:52.006522+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:00:40.116274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:53.103078+0000","last_change":"2024-09-10T18:21:44.213153+0000","last_active":"2024-09-10T18:21:53.103078+0000","last_peered":"2024-09-10T18:21:53.103078+0000","last_clean":"2024-09-10T18:21:53.103078+0000","last_became_active":"2024-09-10T18:21:44.212834+0000","last_became_peered":"2024-09-10T18:21:44.212834+0000","last_unstale":"2024-09-10T18:21:53.103078+0000","last_undegraded":"2024-09-10T18:21:53.103078+0000","last_fullsized":"2024-09-10T18:21:53.103078+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:17:58.381424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:53.103713+0000","last_change":"2024-09-10T18:21:42.188114+0000","last_active":"2024-09-10T18:21:53.103713+0000","last_peered":"2024-09-10T18:21:53.103713+0000","last_clean":"2024-09-10T18:21:53.103713+0000","last_became_active":"2024-09-10T18:21:42.187866+0000","last_became_peered":"2024-09-10T18:21:42.187866+0000","last_unstale":"2024-09-10T18:21:53.103713+0000","last_undegraded":"2024-09-10T18:21:53.103713+0000","last_fullsized":"2024-09-10T18:21:53.103713+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:57.842295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:22:55.686802+0000","last_change":"2024-09-10T18:21:46.223557+0000","last_active":"2024-09-10T18:22:55.686802+0000","last_peered":"2024-09-10T18:22:55.686802+0000","last_clean":"2024-09-10T18:22:55.686802+0000","last_became_active":"2024-09-10T18:21:46.223238+0000","last_became_peered":"2024-09-10T18:21:46.223238+0000","last_unstale":"2024-09-10T18:22:55.686802+0000","last_undegraded":"2024-09-10T18:22:55.686802+0000","last_fullsized":"2024-09-10T18:22:55.686802+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:14:38.870392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.383003+0000","last_change":"2024-09-10T18:21:48.219291+0000","last_active":"2024-09-10T18:21:52.383003+0000","last_peered":"2024-09-10T18:21:52.383003+0000","last_clean":"2024-09-10T18:21:52.383003+0000","last_became_active":"2024-09-10T18:21:48.218997+0000","last_became_peered":"2024-09-10T18:21:48.218997+0000","last_unstale":"2024-09-10T18:21:52.383003+0000","last_undegraded":"2024-09-10T18:21:52.383003+0000","last_fullsized":"2024-09-10T18:21:52.383003+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:37.892083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.006681+0000","last_change":"2024-09-10T18:21:44.212496+0000","last_active":"2024-09-10T18:21:52.006681+0000","last_peered":"2024-09-10T18:21:52.006681+0000","last_clean":"2024-09-10T18:21:52.006681+0000","last_became_active":"2024-09-10T18:21:44.212172+0000","last_became_peered":"2024-09-10T18:21:44.212172+0000","last_unstale":"2024-09-10T18:21:52.006681+0000","last_undegraded":"2024-09-10T18:21:52.006681+0000","last_fullsized":"2024-09-10T18:21:52.006681+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:58:12.578630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.396931+0000","last_change":"2024-09-10T18:21:42.197518+0000","last_active":"2024-09-10T18:21:52.396931+0000","last_peered":"2024-09-10T18:21:52.396931+0000","last_clean":"2024-09-10T18:21:52.396931+0000","last_became_active":"2024-09-10T18:21:42.197278+0000","last_became_peered":"2024-09-10T18:21:42.197278+0000","last_unstale":"2024-09-10T18:21:52.396931+0000","last_undegraded":"2024-09-10T18:21:52.396931+0000","last_fullsized":"2024-09-10T18:21:52.396931+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:12:57.036608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.383182+0000","last_change":"2024-09-10T18:21:46.222424+0000","last_active":"2024-09-10T18:21:52.383182+0000","last_peered":"2024-09-10T18:21:52.383182+0000","last_clean":"2024-09-10T18:21:52.383182+0000","last_became_active":"2024-09-10T18:21:46.221946+0000","last_became_peered":"2024-09-10T18:21:46.221946+0000","last_unstale":"2024-09-10T18:21:52.383182+0000","last_undegraded":"2024-09-10T18:21:52.383182+0000","last_fullsized":"2024-09-10T18:21:52.383182+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:06.041139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.004977+0000","last_change":"2024-09-10T18:21:48.248206+0000","last_active":"2024-09-10T18:21:52.004977+0000","last_peered":"2024-09-10T18:21:52.004977+0000","last_clean":"2024-09-10T18:21:52.004977+0000","last_became_active":"2024-09-10T18:21:48.247899+0000","last_became_peered":"2024-09-10T18:21:48.247899+0000","last_unstale":"2024-09-10T18:21:52.004977+0000","last_undegraded":"2024-09-10T18:21:52.004977+0000","last_fullsized":"2024-09-10T18:21:52.004977+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:48.371158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"59'14","reported_seq":37,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.006107+0000","last_change":"2024-09-10T18:21:44.211671+0000","last_active":"2024-09-10T18:21:52.006107+0000","last_peered":"2024-09-10T18:21:52.006107+0000","last_clean":"2024-09-10T18:21:52.006107+0000","last_became_active":"2024-09-10T18:21:44.211372+0000","last_became_peered":"2024-09-10T18:21:44.211372+0000","last_unstale":"2024-09-10T18:21:52.006107+0000","last_undegraded":"2024-09-10T18:21:52.006107+0000","last_fullsized":"2024-09-10T18:21:52.006107+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:11:54.539741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.019384+0000","last_change":"2024-09-10T18:21:42.195666+0000","last_active":"2024-09-10T18:21:52.019384+0000","last_peered":"2024-09-10T18:21:52.019384+0000","last_clean":"2024-09-10T18:21:52.019384+0000","last_became_active":"2024-09-10T18:21:42.195270+0000","last_became_peered":"2024-09-10T18:21:42.195270+0000","last_unstale":"2024-09-10T18:21:52.019384+0000","last_undegraded":"2024-09-10T18:21:52.019384+0000","last_fullsized":"2024-09-10T18:21:52.019384+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:21.188035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:22:55.686344+0000","last_change":"2024-09-10T18:21:46.219050+0000","last_active":"2024-09-10T18:22:55.686344+0000","last_peered":"2024-09-10T18:22:55.686344+0000","last_clean":"2024-09-10T18:22:55.686344+0000","last_became_active":"2024-09-10T18:21:46.218699+0000","last_became_peered":"2024-09-10T18:21:46.218699+0000","last_unstale":"2024-09-10T18:22:55.686344+0000","last_undegraded":"2024-09-10T18:22:55.686344+0000","last_fullsized":"2024-09-10T18:22:55.686344+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:56.293720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.397302+0000","last_change":"2024-09-10T18:21:48.249560+0000","last_active":"2024-09-10T18:21:52.397302+0000","last_peered":"2024-09-10T18:21:52.397302+0000","last_clean":"2024-09-10T18:21:52.397302+0000","last_became_active":"2024-09-10T18:21:48.249246+0000","last_became_peered":"2024-09-10T18:21:48.249246+0000","last_unstale":"2024-09-10T18:21:52.397302+0000","last_undegraded":"2024-09-10T18:21:52.397302+0000","last_fullsized":"2024-09-10T18:21:52.397302+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:15.844229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"59'18","reported_seq":48,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.004403+0000","last_change":"2024-09-10T18:21:44.544806+0000","last_active":"2024-09-10T18:21:52.004403+0000","last_peered":"2024-09-10T18:21:52.004403+0000","last_clean":"2024-09-10T18:21:52.004403+0000","last_became_active":"2024-09-10T18:21:44.544490+0000","last_became_peered":"2024-09-10T18:21:44.544490+0000","last_unstale":"2024-09-10T18:21:52.004403+0000","last_undegraded":"2024-09-10T18:21:52.004403+0000","last_fullsized":"2024-09-10T18:21:52.004403+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:27.004328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:53.104434+0000","last_change":"2024-09-10T18:21:42.189983+0000","last_active":"2024-09-10T18:21:53.104434+0000","last_peered":"2024-09-10T18:21:53.104434+0000","last_clean":"2024-09-10T18:21:53.104434+0000","last_became_active":"2024-09-10T18:21:42.189725+0000","last_became_peered":"2024-09-10T18:21:42.189725+0000","last_unstale":"2024-09-10T18:21:53.104434+0000","last_undegraded":"2024-09-10T18:21:53.104434+0000","last_fullsized":"2024-09-10T18:21:53.104434+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:21:23.864069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.006028+0000","last_change":"2024-09-10T18:21:46.215416+0000","last_active":"2024-09-10T18:21:52.006028+0000","last_peered":"2024-09-10T18:21:52.006028+0000","last_clean":"2024-09-10T18:21:52.006028+0000","last_became_active":"2024-09-10T18:21:46.215158+0000","last_became_peered":"2024-09-10T18:21:46.215158+0000","last_unstale":"2024-09-10T18:21:52.006028+0000","last_undegraded":"2024-09-10T18:21:52.006028+0000","last_fullsized":"2024-09-10T18:21:52.006028+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:58.888442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.006281+0000","last_change":"2024-09-10T18:21:48.220047+0000","last_active":"2024-09-10T18:21:52.006281+0000","last_peered":"2024-09-10T18:21:52.006281+0000","last_clean":"2024-09-10T18:21:52.006281+0000","last_became_active":"2024-09-10T18:21:48.219746+0000","last_became_peered":"2024-09-10T18:21:48.219746+0000","last_unstale":"2024-09-10T18:21:52.006281+0000","last_undegraded":"2024-09-10T18:21:52.006281+0000","last_fullsized":"2024-09-10T18:21:52.006281+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:05.530770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"59'19","reported_seq":52,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.322610+0000","last_change":"2024-09-10T18:21:44.544653+0000","last_active":"2024-09-10T18:21:52.322610+0000","last_peered":"2024-09-10T18:21:52.322610+0000","last_clean":"2024-09-10T18:21:52.322610+0000","last_became_active":"2024-09-10T18:21:44.544384+0000","last_became_peered":"2024-09-10T18:21:44.544384+0000","last_unstale":"2024-09-10T18:21:52.322610+0000","last_undegraded":"2024-09-10T18:21:52.322610+0000","last_fullsized":"2024-09-10T18:21:52.322610+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:46:34.914351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.397398+0000","last_change":"2024-09-10T18:21:42.197195+0000","last_active":"2024-09-10T18:21:52.397398+0000","last_peered":"2024-09-10T18:21:52.397398+0000","last_clean":"2024-09-10T18:21:52.397398+0000","last_became_active":"2024-09-10T18:21:42.196932+0000","last_became_peered":"2024-09-10T18:21:42.196932+0000","last_unstale":"2024-09-10T18:21:52.397398+0000","last_undegraded":"2024-09-10T18:21:52.397398+0000","last_fullsized":"2024-09-10T18:21:52.397398+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:40.088144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.019269+0000","last_change":"2024-09-10T18:21:46.217608+0000","last_active":"2024-09-10T18:21:52.019269+0000","last_peered":"2024-09-10T18:21:52.019269+0000","last_clean":"2024-09-10T18:21:52.019269+0000","last_became_active":"2024-09-10T18:21:46.217319+0000","last_became_peered":"2024-09-10T18:21:46.217319+0000","last_unstale":"2024-09-10T18:21:52.019269+0000","last_undegraded":"2024-09-10T18:21:52.019269+0000","last_fullsized":"2024-09-10T18:21:52.019269+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:22.721887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:53.103234+0000","last_change":"2024-09-10T18:21:48.248170+0000","last_active":"2024-09-10T18:21:53.103234+0000","last_peered":"2024-09-10T18:21:53.103234+0000","last_clean":"2024-09-10T18:21:53.103234+0000","last_became_active":"2024-09-10T18:21:48.247879+0000","last_became_peered":"2024-09-10T18:21:48.247879+0000","last_unstale":"2024-09-10T18:21:53.103234+0000","last_undegraded":"2024-09-10T18:21:53.103234+0000","last_fullsized":"2024-09-10T18:21:53.103234+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:16:47.722052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:53.102811+0000","last_change":"2024-09-10T18:21:44.215899+0000","last_active":"2024-09-10T18:21:53.102811+0000","last_peered":"2024-09-10T18:21:53.102811+0000","last_clean":"2024-09-10T18:21:53.102811+0000","last_became_active":"2024-09-10T18:21:44.214567+0000","last_became_peered":"2024-09-10T18:21:44.214567+0000","last_unstale":"2024-09-10T18:21:53.102811+0000","last_undegraded":"2024-09-10T18:21:53.102811+0000","last_fullsized":"2024-09-10T18:21:53.102811+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:51.040990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:53.104012+0000","last_change":"2024-09-10T18:21:42.187509+0000","last_active":"2024-09-10T18:21:53.104012+0000","last_peered":"2024-09-10T18:21:53.104012+0000","last_clean":"2024-09-10T18:21:53.104012+0000","last_became_active":"2024-09-10T18:21:42.187265+0000","last_became_peered":"2024-09-10T18:21:42.187265+0000","last_unstale":"2024-09-10T18:21:53.104012+0000","last_undegraded":"2024-09-10T18:21:53.104012+0000","last_fullsized":"2024-09-10T18:21:53.104012+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:35:55.312030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:22:55.686336+0000","last_change":"2024-09-10T18:21:46.216058+0000","last_active":"2024-09-10T18:22:55.686336+0000","last_peered":"2024-09-10T18:22:55.686336+0000","last_clean":"2024-09-10T18:22:55.686336+0000","last_became_active":"2024-09-10T18:21:46.215783+0000","last_became_peered":"2024-09-10T18:21:46.215783+0000","last_unstale":"2024-09-10T18:22:55.686336+0000","last_undegraded":"2024-09-10T18:22:55.686336+0000","last_fullsized":"2024-09-10T18:22:55.686336+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:06.900935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.321885+0000","last_change":"2024-09-10T18:21:48.223222+0000","last_active":"2024-09-10T18:21:52.321885+0000","last_peered":"2024-09-10T18:21:52.321885+0000","last_clean":"2024-09-10T18:21:52.321885+0000","last_became_active":"2024-09-10T18:21:48.222967+0000","last_became_peered":"2024-09-10T18:21:48.222967+0000","last_unstale":"2024-09-10T18:21:52.321885+0000","last_undegraded":"2024-09-10T18:21:52.321885+0000","last_fullsized":"2024-09-10T18:21:52.321885+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:26:53.598813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"59'28","reported_seq":80,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:22:55.686344+0000","last_change":"2024-09-10T18:21:44.219913+0000","last_active":"2024-09-10T18:22:55.686344+0000","last_peered":"2024-09-10T18:22:55.686344+0000","last_clean":"2024-09-10T18:22:55.686344+0000","last_became_active":"2024-09-10T18:21:44.219627+0000","last_became_peered":"2024-09-10T18:21:44.219627+0000","last_unstale":"2024-09-10T18:22:55.686344+0000","last_undegraded":"2024-09-10T18:22:55.686344+0000","last_fullsized":"2024-09-10T18:22:55.686344+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:23:21.885343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.382779+0000","last_change":"2024-09-10T18:21:42.192716+0000","last_active":"2024-09-10T18:21:52.382779+0000","last_peered":"2024-09-10T18:21:52.382779+0000","last_clean":"2024-09-10T18:21:52.382779+0000","last_became_active":"2024-09-10T18:21:42.192352+0000","last_became_peered":"2024-09-10T18:21:42.192352+0000","last_unstale":"2024-09-10T18:21:52.382779+0000","last_undegraded":"2024-09-10T18:21:52.382779+0000","last_fullsized":"2024-09-10T18:21:52.382779+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:29:59.794433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.322576+0000","last_change":"2024-09-10T18:21:46.213595+0000","last_active":"2024-09-10T18:21:52.322576+0000","last_peered":"2024-09-10T18:21:52.322576+0000","last_clean":"2024-09-10T18:21:52.322576+0000","last_became_active":"2024-09-10T18:21:46.212638+0000","last_became_peered":"2024-09-10T18:21:46.212638+0000","last_unstale":"2024-09-10T18:21:52.322576+0000","last_undegraded":"2024-09-10T18:21:52.322576+0000","last_fullsized":"2024-09-10T18:21:52.322576+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:35:49.916818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.005472+0000","last_change":"2024-09-10T18:21:48.222223+0000","last_active":"2024-09-10T18:21:52.005472+0000","last_peered":"2024-09-10T18:21:52.005472+0000","last_clean":"2024-09-10T18:21:52.005472+0000","last_became_active":"2024-09-10T18:21:48.221981+0000","last_became_peered":"2024-09-10T18:21:48.221981+0000","last_unstale":"2024-09-10T18:21:52.005472+0000","last_undegraded":"2024-09-10T18:21:52.005472+0000","last_fullsized":"2024-09-10T18:21:52.005472+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:45:36.100976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"59'13","reported_seq":43,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:53.104535+0000","last_change":"2024-09-10T18:21:44.216116+0000","last_active":"2024-09-10T18:21:53.104535+0000","last_peered":"2024-09-10T18:21:53.104535+0000","last_clean":"2024-09-10T18:21:53.104535+0000","last_became_active":"2024-09-10T18:21:44.215282+0000","last_became_peered":"2024-09-10T18:21:44.215282+0000","last_unstale":"2024-09-10T18:21:53.104535+0000","last_undegraded":"2024-09-10T18:21:53.104535+0000","last_fullsized":"2024-09-10T18:21:53.104535+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:26:55.462477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"60'5","reported_seq":102,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:23:03.819109+0000","last_change":"2024-09-10T18:21:46.203518+0000","last_active":"2024-09-10T18:23:03.819109+0000","last_peered":"2024-09-10T18:23:03.819109+0000","last_clean":"2024-09-10T18:23:03.819109+0000","last_became_active":"2024-09-10T18:21:42.190312+0000","last_became_peered":"2024-09-10T18:21:42.190312+0000","last_unstale":"2024-09-10T18:23:03.819109+0000","last_undegraded":"2024-09-10T18:23:03.819109+0000","last_fullsized":"2024-09-10T18:23:03.819109+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:09:41.138079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051195600000000004,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":76,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.003802+0000","last_change":"2024-09-10T18:21:42.188339+0000","last_active":"2024-09-10T18:21:52.003802+0000","last_peered":"2024-09-10T18:21:52.003802+0000","last_clean":"2024-09-10T18:21:52.003802+0000","last_became_active":"2024-09-10T18:21:42.188062+0000","last_became_peered":"2024-09-10T18:21:42.188062+0000","last_unstale":"2024-09-10T18:21:52.003802+0000","last_undegraded":"2024-09-10T18:21:52.003802+0000","last_fullsized":"2024-09-10T18:21:52.003802+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:22:00.353358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.004133+0000","last_change":"2024-09-10T18:21:46.220317+0000","last_active":"2024-09-10T18:21:52.004133+0000","last_peered":"2024-09-10T18:21:52.004133+0000","last_clean":"2024-09-10T18:21:52.004133+0000","last_became_active":"2024-09-10T18:21:46.220039+0000","last_became_peered":"2024-09-10T18:21:46.220039+0000","last_unstale":"2024-09-10T18:21:52.004133+0000","last_undegraded":"2024-09-10T18:21:52.004133+0000","last_fullsized":"2024-09-10T18:21:52.004133+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:35:18.190468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.397147+0000","last_change":"2024-09-10T18:21:48.249852+0000","last_active":"2024-09-10T18:21:52.397147+0000","last_peered":"2024-09-10T18:21:52.397147+0000","last_clean":"2024-09-10T18:21:52.397147+0000","last_became_active":"2024-09-10T18:21:48.248690+0000","last_became_peered":"2024-09-10T18:21:48.248690+0000","last_unstale":"2024-09-10T18:21:52.397147+0000","last_undegraded":"2024-09-10T18:21:52.397147+0000","last_fullsized":"2024-09-10T18:21:52.397147+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:26:28.559200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"59'12","reported_seq":34,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.321822+0000","last_change":"2024-09-10T18:21:44.218401+0000","last_active":"2024-09-10T18:21:52.321822+0000","last_peered":"2024-09-10T18:21:52.321822+0000","last_clean":"2024-09-10T18:21:52.321822+0000","last_became_active":"2024-09-10T18:21:44.218026+0000","last_became_peered":"2024-09-10T18:21:44.218026+0000","last_unstale":"2024-09-10T18:21:52.321822+0000","last_undegraded":"2024-09-10T18:21:52.321822+0000","last_fullsized":"2024-09-10T18:21:52.321822+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:53.692887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"54'1","reported_seq":24,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.322233+0000","last_change":"2024-09-10T18:21:46.204896+0000","last_active":"2024-09-10T18:21:52.322233+0000","last_peered":"2024-09-10T18:21:52.322233+0000","last_clean":"2024-09-10T18:21:52.322233+0000","last_became_active":"2024-09-10T18:21:42.193041+0000","last_became_peered":"2024-09-10T18:21:42.193041+0000","last_unstale":"2024-09-10T18:21:52.322233+0000","last_undegraded":"2024-09-10T18:21:52.322233+0000","last_fullsized":"2024-09-10T18:21:52.322233+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:38.003652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098752999999999992,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.396985+0000","last_change":"2024-09-10T18:21:42.197813+0000","last_active":"2024-09-10T18:21:52.396985+0000","last_peered":"2024-09-10T18:21:52.396985+0000","last_clean":"2024-09-10T18:21:52.396985+0000","last_became_active":"2024-09-10T18:21:42.197499+0000","last_became_peered":"2024-09-10T18:21:42.197499+0000","last_unstale":"2024-09-10T18:21:52.396985+0000","last_undegraded":"2024-09-10T18:21:52.396985+0000","last_fullsized":"2024-09-10T18:21:52.396985+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:32.117794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.383218+0000","last_change":"2024-09-10T18:21:46.218067+0000","last_active":"2024-09-10T18:21:52.383218+0000","last_peered":"2024-09-10T18:21:52.383218+0000","last_clean":"2024-09-10T18:21:52.383218+0000","last_became_active":"2024-09-10T18:21:46.214073+0000","last_became_peered":"2024-09-10T18:21:46.214073+0000","last_unstale":"2024-09-10T18:21:52.383218+0000","last_undegraded":"2024-09-10T18:21:52.383218+0000","last_fullsized":"2024-09-10T18:21:52.383218+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:57:48.766455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:53.103896+0000","last_change":"2024-09-10T18:21:48.221395+0000","last_active":"2024-09-10T18:21:53.103896+0000","last_peered":"2024-09-10T18:21:53.103896+0000","last_clean":"2024-09-10T18:21:53.103896+0000","last_became_active":"2024-09-10T18:21:48.221130+0000","last_became_peered":"2024-09-10T18:21:48.221130+0000","last_unstale":"2024-09-10T18:21:53.103896+0000","last_undegraded":"2024-09-10T18:21:53.103896+0000","last_fullsized":"2024-09-10T18:21:53.103896+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:21.546806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"59'14","reported_seq":38,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.019313+0000","last_change":"2024-09-10T18:21:44.545097+0000","last_active":"2024-09-10T18:21:52.019313+0000","last_peered":"2024-09-10T18:21:52.019313+0000","last_clean":"2024-09-10T18:21:52.019313+0000","last_became_active":"2024-09-10T18:21:44.544597+0000","last_became_peered":"2024-09-10T18:21:44.544597+0000","last_unstale":"2024-09-10T18:21:52.019313+0000","last_undegraded":"2024-09-10T18:21:52.019313+0000","last_fullsized":"2024-09-10T18:21:52.019313+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:54:28.205142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'2","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.005251+0000","last_change":"2024-09-10T18:21:46.209528+0000","last_active":"2024-09-10T18:21:52.005251+0000","last_peered":"2024-09-10T18:21:52.005251+0000","last_clean":"2024-09-10T18:21:52.005251+0000","last_became_active":"2024-09-10T18:21:42.190380+0000","last_became_peered":"2024-09-10T18:21:42.190380+0000","last_unstale":"2024-09-10T18:21:52.005251+0000","last_undegraded":"2024-09-10T18:21:52.005251+0000","last_fullsized":"2024-09-10T18:21:52.005251+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:15:02.719844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094658199999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.382880+0000","last_change":"2024-09-10T18:21:42.195366+0000","last_active":"2024-09-10T18:21:52.382880+0000","last_peered":"2024-09-10T18:21:52.382880+0000","last_clean":"2024-09-10T18:21:52.382880+0000","last_became_active":"2024-09-10T18:21:42.195095+0000","last_became_peered":"2024-09-10T18:21:42.195095+0000","last_unstale":"2024-09-10T18:21:52.382880+0000","last_undegraded":"2024-09-10T18:21:52.382880+0000","last_fullsized":"2024-09-10T18:21:52.382880+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:24:18.917978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"61'79","reported_seq":52,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:22:21.090419+0000","last_change":"2024-09-10T18:21:31.484955+0000","last_active":"2024-09-10T18:22:21.090419+0000","last_peered":"2024-09-10T18:22:21.090419+0000","last_clean":"2024-09-10T18:22:21.090419+0000","last_became_active":"2024-09-10T18:21:31.477446+0000","last_became_peered":"2024-09-10T18:21:31.477446+0000","last_unstale":"2024-09-10T18:22:21.090419+0000","last_undegraded":"2024-09-10T18:22:21.090419+0000","last_fullsized":"2024-09-10T18:22:21.090419+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:19:32.548162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:19:32.548162+0000","last_clean_scrub_stamp":"2024-09-10T18:19:32.548162+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:04.128475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.397055+0000","last_change":"2024-09-10T18:21:46.218393+0000","last_active":"2024-09-10T18:21:52.397055+0000","last_peered":"2024-09-10T18:21:52.397055+0000","last_clean":"2024-09-10T18:21:52.397055+0000","last_became_active":"2024-09-10T18:21:46.218107+0000","last_became_peered":"2024-09-10T18:21:46.218107+0000","last_unstale":"2024-09-10T18:21:52.397055+0000","last_undegraded":"2024-09-10T18:21:52.397055+0000","last_fullsized":"2024-09-10T18:21:52.397055+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:36:11.032542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.383250+0000","last_change":"2024-09-10T18:21:48.225584+0000","last_active":"2024-09-10T18:21:52.383250+0000","last_peered":"2024-09-10T18:21:52.383250+0000","last_clean":"2024-09-10T18:21:52.383250+0000","last_became_active":"2024-09-10T18:21:48.225234+0000","last_became_peered":"2024-09-10T18:21:48.225234+0000","last_unstale":"2024-09-10T18:21:52.383250+0000","last_undegraded":"2024-09-10T18:21:52.383250+0000","last_fullsized":"2024-09-10T18:21:52.383250+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:31.131019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"59'19","reported_seq":47,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.019160+0000","last_change":"2024-09-10T18:21:44.544859+0000","last_active":"2024-09-10T18:21:52.019160+0000","last_peered":"2024-09-10T18:21:52.019160+0000","last_clean":"2024-09-10T18:21:52.019160+0000","last_became_active":"2024-09-10T18:21:44.544505+0000","last_became_peered":"2024-09-10T18:21:44.544505+0000","last_unstale":"2024-09-10T18:21:52.019160+0000","last_undegraded":"2024-09-10T18:21:52.019160+0000","last_fullsized":"2024-09-10T18:21:52.019160+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:35:28.420372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.004782+0000","last_change":"2024-09-10T18:21:42.190925+0000","last_active":"2024-09-10T18:21:52.004782+0000","last_peered":"2024-09-10T18:21:52.004782+0000","last_clean":"2024-09-10T18:21:52.004782+0000","last_became_active":"2024-09-10T18:21:42.190660+0000","last_became_peered":"2024-09-10T18:21:42.190660+0000","last_unstale":"2024-09-10T18:21:52.004782+0000","last_undegraded":"2024-09-10T18:21:52.004782+0000","last_fullsized":"2024-09-10T18:21:52.004782+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:24:24.760672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.004046+0000","last_change":"2024-09-10T18:21:46.218021+0000","last_active":"2024-09-10T18:21:52.004046+0000","last_peered":"2024-09-10T18:21:52.004046+0000","last_clean":"2024-09-10T18:21:52.004046+0000","last_became_active":"2024-09-10T18:21:46.217382+0000","last_became_peered":"2024-09-10T18:21:46.217382+0000","last_unstale":"2024-09-10T18:21:52.004046+0000","last_undegraded":"2024-09-10T18:21:52.004046+0000","last_fullsized":"2024-09-10T18:21:52.004046+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:27:17.458917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.397042+0000","last_change":"2024-09-10T18:21:48.225762+0000","last_active":"2024-09-10T18:21:52.397042+0000","last_peered":"2024-09-10T18:21:52.397042+0000","last_clean":"2024-09-10T18:21:52.397042+0000","last_became_active":"2024-09-10T18:21:48.225273+0000","last_became_peered":"2024-09-10T18:21:48.225273+0000","last_unstale":"2024-09-10T18:21:52.397042+0000","last_undegraded":"2024-09-10T18:21:52.397042+0000","last_fullsized":"2024-09-10T18:21:52.397042+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:00.622136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.321816+0000","last_change":"2024-09-10T18:21:44.219194+0000","last_active":"2024-09-10T18:21:52.321816+0000","last_peered":"2024-09-10T18:21:52.321816+0000","last_clean":"2024-09-10T18:21:52.321816+0000","last_became_active":"2024-09-10T18:21:44.218208+0000","last_became_peered":"2024-09-10T18:21:44.218208+0000","last_unstale":"2024-09-10T18:21:52.321816+0000","last_undegraded":"2024-09-10T18:21:52.321816+0000","last_fullsized":"2024-09-10T18:21:52.321816+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:23.592157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:53.104335+0000","last_change":"2024-09-10T18:21:42.186979+0000","last_active":"2024-09-10T18:21:53.104335+0000","last_peered":"2024-09-10T18:21:53.104335+0000","last_clean":"2024-09-10T18:21:53.104335+0000","last_became_active":"2024-09-10T18:21:42.186715+0000","last_became_peered":"2024-09-10T18:21:42.186715+0000","last_unstale":"2024-09-10T18:21:53.104335+0000","last_undegraded":"2024-09-10T18:21:53.104335+0000","last_fullsized":"2024-09-10T18:21:53.104335+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:16:27.222177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.003898+0000","last_change":"2024-09-10T18:21:46.213893+0000","last_active":"2024-09-10T18:21:52.003898+0000","last_peered":"2024-09-10T18:21:52.003898+0000","last_clean":"2024-09-10T18:21:52.003898+0000","last_became_active":"2024-09-10T18:21:46.213635+0000","last_became_peered":"2024-09-10T18:21:46.213635+0000","last_unstale":"2024-09-10T18:21:52.003898+0000","last_undegraded":"2024-09-10T18:21:52.003898+0000","last_fullsized":"2024-09-10T18:21:52.003898+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:10:35.774133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.322083+0000","last_change":"2024-09-10T18:21:48.222057+0000","last_active":"2024-09-10T18:21:52.322083+0000","last_peered":"2024-09-10T18:21:52.322083+0000","last_clean":"2024-09-10T18:21:52.322083+0000","last_became_active":"2024-09-10T18:21:48.221787+0000","last_became_peered":"2024-09-10T18:21:48.221787+0000","last_unstale":"2024-09-10T18:21:52.322083+0000","last_undegraded":"2024-09-10T18:21:52.322083+0000","last_fullsized":"2024-09-10T18:21:52.322083+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:10.803275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.382496+0000","last_change":"2024-09-10T18:21:44.543735+0000","last_active":"2024-09-10T18:21:52.382496+0000","last_peered":"2024-09-10T18:21:52.382496+0000","last_clean":"2024-09-10T18:21:52.382496+0000","last_became_active":"2024-09-10T18:21:44.543366+0000","last_became_peered":"2024-09-10T18:21:44.543366+0000","last_unstale":"2024-09-10T18:21:52.382496+0000","last_undegraded":"2024-09-10T18:21:52.382496+0000","last_fullsized":"2024-09-10T18:21:52.382496+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:12:02.537666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"52'1","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.004348+0000","last_change":"2024-09-10T18:21:42.192027+0000","last_active":"2024-09-10T18:21:52.004348+0000","last_peered":"2024-09-10T18:21:52.004348+0000","last_clean":"2024-09-10T18:21:52.004348+0000","last_became_active":"2024-09-10T18:21:42.191768+0000","last_became_peered":"2024-09-10T18:21:42.191768+0000","last_unstale":"2024-09-10T18:21:52.004348+0000","last_undegraded":"2024-09-10T18:21:52.004348+0000","last_fullsized":"2024-09-10T18:21:52.004348+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:00.027576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:22:55.686702+0000","last_change":"2024-09-10T18:21:46.219325+0000","last_active":"2024-09-10T18:22:55.686702+0000","last_peered":"2024-09-10T18:22:55.686702+0000","last_clean":"2024-09-10T18:22:55.686702+0000","last_became_active":"2024-09-10T18:21:46.218717+0000","last_became_peered":"2024-09-10T18:21:46.218717+0000","last_unstale":"2024-09-10T18:22:55.686702+0000","last_undegraded":"2024-09-10T18:22:55.686702+0000","last_fullsized":"2024-09-10T18:22:55.686702+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:49:23.946749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.382615+0000","last_change":"2024-09-10T18:21:48.249077+0000","last_active":"2024-09-10T18:21:52.382615+0000","last_peered":"2024-09-10T18:21:52.382615+0000","last_clean":"2024-09-10T18:21:52.382615+0000","last_became_active":"2024-09-10T18:21:48.248808+0000","last_became_peered":"2024-09-10T18:21:48.248808+0000","last_unstale":"2024-09-10T18:21:52.382615+0000","last_undegraded":"2024-09-10T18:21:52.382615+0000","last_fullsized":"2024-09-10T18:21:52.382615+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:00:31.037243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"59'12","reported_seq":39,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.006053+0000","last_change":"2024-09-10T18:21:44.212445+0000","last_active":"2024-09-10T18:21:52.006053+0000","last_peered":"2024-09-10T18:21:52.006053+0000","last_clean":"2024-09-10T18:21:52.006053+0000","last_became_active":"2024-09-10T18:21:44.212098+0000","last_became_peered":"2024-09-10T18:21:44.212098+0000","last_unstale":"2024-09-10T18:21:52.006053+0000","last_undegraded":"2024-09-10T18:21:52.006053+0000","last_fullsized":"2024-09-10T18:21:52.006053+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:37:40.320627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"52'2","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.006738+0000","last_change":"2024-09-10T18:21:42.187924+0000","last_active":"2024-09-10T18:21:52.006738+0000","last_peered":"2024-09-10T18:21:52.006738+0000","last_clean":"2024-09-10T18:21:52.006738+0000","last_became_active":"2024-09-10T18:21:42.187638+0000","last_became_peered":"2024-09-10T18:21:42.187638+0000","last_unstale":"2024-09-10T18:21:52.006738+0000","last_undegraded":"2024-09-10T18:21:52.006738+0000","last_fullsized":"2024-09-10T18:21:52.006738+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:48:21.137550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.003675+0000","last_change":"2024-09-10T18:21:46.213109+0000","last_active":"2024-09-10T18:21:52.003675+0000","last_peered":"2024-09-10T18:21:52.003675+0000","last_clean":"2024-09-10T18:21:52.003675+0000","last_became_active":"2024-09-10T18:21:46.212841+0000","last_became_peered":"2024-09-10T18:21:46.212841+0000","last_unstale":"2024-09-10T18:21:52.003675+0000","last_undegraded":"2024-09-10T18:21:52.003675+0000","last_fullsized":"2024-09-10T18:21:52.003675+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:35:20.777465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.004970+0000","last_change":"2024-09-10T18:21:48.221562+0000","last_active":"2024-09-10T18:21:52.004970+0000","last_peered":"2024-09-10T18:21:52.004970+0000","last_clean":"2024-09-10T18:21:52.004970+0000","last_became_active":"2024-09-10T18:21:48.221314+0000","last_became_peered":"2024-09-10T18:21:48.221314+0000","last_unstale":"2024-09-10T18:21:52.004970+0000","last_undegraded":"2024-09-10T18:21:52.004970+0000","last_fullsized":"2024-09-10T18:21:52.004970+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:27:34.121104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.322819+0000","last_change":"2024-09-10T18:21:44.543784+0000","last_active":"2024-09-10T18:21:52.322819+0000","last_peered":"2024-09-10T18:21:52.322819+0000","last_clean":"2024-09-10T18:21:52.322819+0000","last_became_active":"2024-09-10T18:21:44.543498+0000","last_became_peered":"2024-09-10T18:21:44.543498+0000","last_unstale":"2024-09-10T18:21:52.322819+0000","last_undegraded":"2024-09-10T18:21:52.322819+0000","last_fullsized":"2024-09-10T18:21:52.322819+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:40:42.061165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.003798+0000","last_change":"2024-09-10T18:21:42.186885+0000","last_active":"2024-09-10T18:21:52.003798+0000","last_peered":"2024-09-10T18:21:52.003798+0000","last_clean":"2024-09-10T18:21:52.003798+0000","last_became_active":"2024-09-10T18:21:42.186604+0000","last_became_peered":"2024-09-10T18:21:42.186604+0000","last_unstale":"2024-09-10T18:21:52.003798+0000","last_undegraded":"2024-09-10T18:21:52.003798+0000","last_fullsized":"2024-09-10T18:21:52.003798+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:05:28.202909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.005598+0000","last_change":"2024-09-10T18:21:46.218222+0000","last_active":"2024-09-10T18:21:52.005598+0000","last_peered":"2024-09-10T18:21:52.005598+0000","last_clean":"2024-09-10T18:21:52.005598+0000","last_became_active":"2024-09-10T18:21:46.217986+0000","last_became_peered":"2024-09-10T18:21:46.217986+0000","last_unstale":"2024-09-10T18:21:52.005598+0000","last_undegraded":"2024-09-10T18:21:52.005598+0000","last_fullsized":"2024-09-10T18:21:52.005598+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:05:27.223317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.004322+0000","last_change":"2024-09-10T18:21:48.216201+0000","last_active":"2024-09-10T18:21:52.004322+0000","last_peered":"2024-09-10T18:21:52.004322+0000","last_clean":"2024-09-10T18:21:52.004322+0000","last_became_active":"2024-09-10T18:21:48.215951+0000","last_became_peered":"2024-09-10T18:21:48.215951+0000","last_unstale":"2024-09-10T18:21:52.004322+0000","last_undegraded":"2024-09-10T18:21:52.004322+0000","last_fullsized":"2024-09-10T18:21:52.004322+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:49:46.528068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.019019+0000","last_change":"2024-09-10T18:21:42.193990+0000","last_active":"2024-09-10T18:21:52.019019+0000","last_peered":"2024-09-10T18:21:52.019019+0000","last_clean":"2024-09-10T18:21:52.019019+0000","last_became_active":"2024-09-10T18:21:42.193479+0000","last_became_peered":"2024-09-10T18:21:42.193479+0000","last_unstale":"2024-09-10T18:21:52.019019+0000","last_undegraded":"2024-09-10T18:21:52.019019+0000","last_fullsized":"2024-09-10T18:21:52.019019+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:17.563667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"59'6","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.005842+0000","last_change":"2024-09-10T18:21:44.544793+0000","last_active":"2024-09-10T18:21:52.005842+0000","last_peered":"2024-09-10T18:21:52.005842+0000","last_clean":"2024-09-10T18:21:52.005842+0000","last_became_active":"2024-09-10T18:21:44.544457+0000","last_became_peered":"2024-09-10T18:21:44.544457+0000","last_unstale":"2024-09-10T18:21:52.005842+0000","last_undegraded":"2024-09-10T18:21:52.005842+0000","last_fullsized":"2024-09-10T18:21:52.005842+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:40:09.752979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.383141+0000","last_change":"2024-09-10T18:21:46.215678+0000","last_active":"2024-09-10T18:21:52.383141+0000","last_peered":"2024-09-10T18:21:52.383141+0000","last_clean":"2024-09-10T18:21:52.383141+0000","last_became_active":"2024-09-10T18:21:46.215365+0000","last_became_peered":"2024-09-10T18:21:46.215365+0000","last_unstale":"2024-09-10T18:21:52.383141+0000","last_undegraded":"2024-09-10T18:21:52.383141+0000","last_fullsized":"2024-09-10T18:21:52.383141+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:05.489735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.397137+0000","last_change":"2024-09-10T18:21:48.249927+0000","last_active":"2024-09-10T18:21:52.397137+0000","last_peered":"2024-09-10T18:21:52.397137+0000","last_clean":"2024-09-10T18:21:52.397137+0000","last_became_active":"2024-09-10T18:21:48.249663+0000","last_became_peered":"2024-09-10T18:21:48.249663+0000","last_unstale":"2024-09-10T18:21:52.397137+0000","last_undegraded":"2024-09-10T18:21:52.397137+0000","last_fullsized":"2024-09-10T18:21:52.397137+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:33.485123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.005245+0000","last_change":"2024-09-10T18:21:44.543239+0000","last_active":"2024-09-10T18:21:52.005245+0000","last_peered":"2024-09-10T18:21:52.005245+0000","last_clean":"2024-09-10T18:21:52.005245+0000","last_became_active":"2024-09-10T18:21:44.542923+0000","last_became_peered":"2024-09-10T18:21:44.542923+0000","last_unstale":"2024-09-10T18:21:52.005245+0000","last_undegraded":"2024-09-10T18:21:52.005245+0000","last_fullsized":"2024-09-10T18:21:52.005245+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:03:40.909083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.382781+0000","last_change":"2024-09-10T18:21:42.193818+0000","last_active":"2024-09-10T18:21:52.382781+0000","last_peered":"2024-09-10T18:21:52.382781+0000","last_clean":"2024-09-10T18:21:52.382781+0000","last_became_active":"2024-09-10T18:21:42.193532+0000","last_became_peered":"2024-09-10T18:21:42.193532+0000","last_unstale":"2024-09-10T18:21:52.382781+0000","last_undegraded":"2024-09-10T18:21:52.382781+0000","last_fullsized":"2024-09-10T18:21:52.382781+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:03:04.235353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:22:55.686769+0000","last_change":"2024-09-10T18:21:46.215394+0000","last_active":"2024-09-10T18:22:55.686769+0000","last_peered":"2024-09-10T18:22:55.686769+0000","last_clean":"2024-09-10T18:22:55.686769+0000","last_became_active":"2024-09-10T18:21:46.215063+0000","last_became_peered":"2024-09-10T18:21:46.215063+0000","last_unstale":"2024-09-10T18:22:55.686769+0000","last_undegraded":"2024-09-10T18:22:55.686769+0000","last_fullsized":"2024-09-10T18:22:55.686769+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:45:37.828828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.322586+0000","last_change":"2024-09-10T18:21:48.223251+0000","last_active":"2024-09-10T18:21:52.322586+0000","last_peered":"2024-09-10T18:21:52.322586+0000","last_clean":"2024-09-10T18:21:52.322586+0000","last_became_active":"2024-09-10T18:21:48.222995+0000","last_became_peered":"2024-09-10T18:21:48.222995+0000","last_unstale":"2024-09-10T18:21:52.322586+0000","last_undegraded":"2024-09-10T18:21:52.322586+0000","last_fullsized":"2024-09-10T18:21:52.322586+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:09:02.531330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.383097+0000","last_change":"2024-09-10T18:21:44.544958+0000","last_active":"2024-09-10T18:21:52.383097+0000","last_peered":"2024-09-10T18:21:52.383097+0000","last_clean":"2024-09-10T18:21:52.383097+0000","last_became_active":"2024-09-10T18:21:44.544654+0000","last_became_peered":"2024-09-10T18:21:44.544654+0000","last_unstale":"2024-09-10T18:21:52.383097+0000","last_undegraded":"2024-09-10T18:21:52.383097+0000","last_fullsized":"2024-09-10T18:21:52.383097+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:56:46.447734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.322781+0000","last_change":"2024-09-10T18:21:42.195177+0000","last_active":"2024-09-10T18:21:52.322781+0000","last_peered":"2024-09-10T18:21:52.322781+0000","last_clean":"2024-09-10T18:21:52.322781+0000","last_became_active":"2024-09-10T18:21:42.194408+0000","last_became_peered":"2024-09-10T18:21:42.194408+0000","last_unstale":"2024-09-10T18:21:52.322781+0000","last_undegraded":"2024-09-10T18:21:52.322781+0000","last_fullsized":"2024-09-10T18:21:52.322781+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:36.889116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:22:55.686405+0000","last_change":"2024-09-10T18:21:46.222267+0000","last_active":"2024-09-10T18:22:55.686405+0000","last_peered":"2024-09-10T18:22:55.686405+0000","last_clean":"2024-09-10T18:22:55.686405+0000","last_became_active":"2024-09-10T18:21:46.221973+0000","last_became_peered":"2024-09-10T18:21:46.221973+0000","last_unstale":"2024-09-10T18:22:55.686405+0000","last_undegraded":"2024-09-10T18:22:55.686405+0000","last_fullsized":"2024-09-10T18:22:55.686405+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:18:19.797607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.006565+0000","last_change":"2024-09-10T18:21:48.216075+0000","last_active":"2024-09-10T18:21:52.006565+0000","last_peered":"2024-09-10T18:21:52.006565+0000","last_clean":"2024-09-10T18:21:52.006565+0000","last_became_active":"2024-09-10T18:21:48.215762+0000","last_became_peered":"2024-09-10T18:21:48.215762+0000","last_unstale":"2024-09-10T18:21:52.006565+0000","last_undegraded":"2024-09-10T18:21:52.006565+0000","last_fullsized":"2024-09-10T18:21:52.006565+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:33.770854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:53.102812+0000","last_change":"2024-09-10T18:21:44.216152+0000","last_active":"2024-09-10T18:21:53.102812+0000","last_peered":"2024-09-10T18:21:53.102812+0000","last_clean":"2024-09-10T18:21:53.102812+0000","last_became_active":"2024-09-10T18:21:44.215371+0000","last_became_peered":"2024-09-10T18:21:44.215371+0000","last_unstale":"2024-09-10T18:21:53.102812+0000","last_undegraded":"2024-09-10T18:21:53.102812+0000","last_fullsized":"2024-09-10T18:21:53.102812+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:13.942356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.019180+0000","last_change":"2024-09-10T18:21:42.193635+0000","last_active":"2024-09-10T18:21:52.019180+0000","last_peered":"2024-09-10T18:21:52.019180+0000","last_clean":"2024-09-10T18:21:52.019180+0000","last_became_active":"2024-09-10T18:21:42.193319+0000","last_became_peered":"2024-09-10T18:21:42.193319+0000","last_unstale":"2024-09-10T18:21:52.019180+0000","last_undegraded":"2024-09-10T18:21:52.019180+0000","last_fullsized":"2024-09-10T18:21:52.019180+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:41:03.510218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.005050+0000","last_change":"2024-09-10T18:21:46.219503+0000","last_active":"2024-09-10T18:21:52.005050+0000","last_peered":"2024-09-10T18:21:52.005050+0000","last_clean":"2024-09-10T18:21:52.005050+0000","last_became_active":"2024-09-10T18:21:46.219222+0000","last_became_peered":"2024-09-10T18:21:46.219222+0000","last_unstale":"2024-09-10T18:21:52.005050+0000","last_undegraded":"2024-09-10T18:21:52.005050+0000","last_fullsized":"2024-09-10T18:21:52.005050+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:25.256348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.321860+0000","last_change":"2024-09-10T18:21:48.225120+0000","last_active":"2024-09-10T18:21:52.321860+0000","last_peered":"2024-09-10T18:21:52.321860+0000","last_clean":"2024-09-10T18:21:52.321860+0000","last_became_active":"2024-09-10T18:21:48.224860+0000","last_became_peered":"2024-09-10T18:21:48.224860+0000","last_unstale":"2024-09-10T18:21:52.321860+0000","last_undegraded":"2024-09-10T18:21:52.321860+0000","last_fullsized":"2024-09-10T18:21:52.321860+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:44:12.980814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.006312+0000","last_change":"2024-09-10T18:21:44.213806+0000","last_active":"2024-09-10T18:21:52.006312+0000","last_peered":"2024-09-10T18:21:52.006312+0000","last_clean":"2024-09-10T18:21:52.006312+0000","last_became_active":"2024-09-10T18:21:44.213496+0000","last_became_peered":"2024-09-10T18:21:44.213496+0000","last_unstale":"2024-09-10T18:21:52.006312+0000","last_undegraded":"2024-09-10T18:21:52.006312+0000","last_fullsized":"2024-09-10T18:21:52.006312+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:52.815568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:53.104340+0000","last_change":"2024-09-10T18:21:42.189078+0000","last_active":"2024-09-10T18:21:53.104340+0000","last_peered":"2024-09-10T18:21:53.104340+0000","last_clean":"2024-09-10T18:21:53.104340+0000","last_became_active":"2024-09-10T18:21:42.188150+0000","last_became_peered":"2024-09-10T18:21:42.188150+0000","last_unstale":"2024-09-10T18:21:53.104340+0000","last_undegraded":"2024-09-10T18:21:53.104340+0000","last_fullsized":"2024-09-10T18:21:53.104340+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:17:05.659061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:53.103629+0000","last_change":"2024-09-10T18:21:46.216514+0000","last_active":"2024-09-10T18:21:53.103629+0000","last_peered":"2024-09-10T18:21:53.103629+0000","last_clean":"2024-09-10T18:21:53.103629+0000","last_became_active":"2024-09-10T18:21:46.215932+0000","last_became_peered":"2024-09-10T18:21:46.215932+0000","last_unstale":"2024-09-10T18:21:53.103629+0000","last_undegraded":"2024-09-10T18:21:53.103629+0000","last_fullsized":"2024-09-10T18:21:53.103629+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:08:13.255160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.004276+0000","last_change":"2024-09-10T18:21:48.224686+0000","last_active":"2024-09-10T18:21:52.004276+0000","last_peered":"2024-09-10T18:21:52.004276+0000","last_clean":"2024-09-10T18:21:52.004276+0000","last_became_active":"2024-09-10T18:21:48.224423+0000","last_became_peered":"2024-09-10T18:21:48.224423+0000","last_unstale":"2024-09-10T18:21:52.004276+0000","last_undegraded":"2024-09-10T18:21:52.004276+0000","last_fullsized":"2024-09-10T18:21:52.004276+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:16.684266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"59'4","reported_seq":22,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.004317+0000","last_change":"2024-09-10T18:21:44.218991+0000","last_active":"2024-09-10T18:21:52.004317+0000","last_peered":"2024-09-10T18:21:52.004317+0000","last_clean":"2024-09-10T18:21:52.004317+0000","last_became_active":"2024-09-10T18:21:44.218658+0000","last_became_peered":"2024-09-10T18:21:44.218658+0000","last_unstale":"2024-09-10T18:21:52.004317+0000","last_undegraded":"2024-09-10T18:21:52.004317+0000","last_fullsized":"2024-09-10T18:21:52.004317+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:39:28.896129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.383201+0000","last_change":"2024-09-10T18:21:42.193238+0000","last_active":"2024-09-10T18:21:52.383201+0000","last_peered":"2024-09-10T18:21:52.383201+0000","last_clean":"2024-09-10T18:21:52.383201+0000","last_became_active":"2024-09-10T18:21:42.192955+0000","last_became_peered":"2024-09-10T18:21:42.192955+0000","last_unstale":"2024-09-10T18:21:52.383201+0000","last_undegraded":"2024-09-10T18:21:52.383201+0000","last_fullsized":"2024-09-10T18:21:52.383201+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:00:41.711996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.018874+0000","last_change":"2024-09-10T18:21:46.214983+0000","last_active":"2024-09-10T18:21:52.018874+0000","last_peered":"2024-09-10T18:21:52.018874+0000","last_clean":"2024-09-10T18:21:52.018874+0000","last_became_active":"2024-09-10T18:21:46.214542+0000","last_became_peered":"2024-09-10T18:21:46.214542+0000","last_unstale":"2024-09-10T18:21:52.018874+0000","last_undegraded":"2024-09-10T18:21:52.018874+0000","last_fullsized":"2024-09-10T18:21:52.018874+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:44.015035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.396896+0000","last_change":"2024-09-10T18:21:48.216479+0000","last_active":"2024-09-10T18:21:52.396896+0000","last_peered":"2024-09-10T18:21:52.396896+0000","last_clean":"2024-09-10T18:21:52.396896+0000","last_became_active":"2024-09-10T18:21:48.216204+0000","last_became_peered":"2024-09-10T18:21:48.216204+0000","last_unstale":"2024-09-10T18:21:52.396896+0000","last_undegraded":"2024-09-10T18:21:52.396896+0000","last_fullsized":"2024-09-10T18:21:52.396896+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:17:30.545376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.004576+0000","last_change":"2024-09-10T18:21:44.544622+0000","last_active":"2024-09-10T18:21:52.004576+0000","last_peered":"2024-09-10T18:21:52.004576+0000","last_clean":"2024-09-10T18:21:52.004576+0000","last_became_active":"2024-09-10T18:21:44.544384+0000","last_became_peered":"2024-09-10T18:21:44.544384+0000","last_unstale":"2024-09-10T18:21:52.004576+0000","last_undegraded":"2024-09-10T18:21:52.004576+0000","last_fullsized":"2024-09-10T18:21:52.004576+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:57:03.121884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.019367+0000","last_change":"2024-09-10T18:21:42.191810+0000","last_active":"2024-09-10T18:21:52.019367+0000","last_peered":"2024-09-10T18:21:52.019367+0000","last_clean":"2024-09-10T18:21:52.019367+0000","last_became_active":"2024-09-10T18:21:42.191560+0000","last_became_peered":"2024-09-10T18:21:42.191560+0000","last_unstale":"2024-09-10T18:21:52.019367+0000","last_undegraded":"2024-09-10T18:21:52.019367+0000","last_fullsized":"2024-09-10T18:21:52.019367+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:35:20.356744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.396782+0000","last_change":"2024-09-10T18:21:46.217609+0000","last_active":"2024-09-10T18:21:52.396782+0000","last_peered":"2024-09-10T18:21:52.396782+0000","last_clean":"2024-09-10T18:21:52.396782+0000","last_became_active":"2024-09-10T18:21:46.217313+0000","last_became_peered":"2024-09-10T18:21:46.217313+0000","last_unstale":"2024-09-10T18:21:52.396782+0000","last_undegraded":"2024-09-10T18:21:52.396782+0000","last_fullsized":"2024-09-10T18:21:52.396782+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:31:45.563334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.004839+0000","last_change":"2024-09-10T18:21:48.249570+0000","last_active":"2024-09-10T18:21:52.004839+0000","last_peered":"2024-09-10T18:21:52.004839+0000","last_clean":"2024-09-10T18:21:52.004839+0000","last_became_active":"2024-09-10T18:21:48.249257+0000","last_became_peered":"2024-09-10T18:21:48.249257+0000","last_unstale":"2024-09-10T18:21:52.004839+0000","last_undegraded":"2024-09-10T18:21:52.004839+0000","last_fullsized":"2024-09-10T18:21:52.004839+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:55.391437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.397041+0000","last_change":"2024-09-10T18:21:48.215302+0000","last_active":"2024-09-10T18:21:52.397041+0000","last_peered":"2024-09-10T18:21:52.397041+0000","last_clean":"2024-09-10T18:21:52.397041+0000","last_became_active":"2024-09-10T18:21:48.214978+0000","last_became_peered":"2024-09-10T18:21:48.214978+0000","last_unstale":"2024-09-10T18:21:52.397041+0000","last_undegraded":"2024-09-10T18:21:52.397041+0000","last_fullsized":"2024-09-10T18:21:52.397041+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:57:22.002791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.382568+0000","last_change":"2024-09-10T18:21:42.192531+0000","last_active":"2024-09-10T18:21:52.382568+0000","last_peered":"2024-09-10T18:21:52.382568+0000","last_clean":"2024-09-10T18:21:52.382568+0000","last_became_active":"2024-09-10T18:21:42.190963+0000","last_became_peered":"2024-09-10T18:21:42.190963+0000","last_unstale":"2024-09-10T18:21:52.382568+0000","last_undegraded":"2024-09-10T18:21:52.382568+0000","last_fullsized":"2024-09-10T18:21:52.382568+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:30:14.112856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.322780+0000","last_change":"2024-09-10T18:21:44.544740+0000","last_active":"2024-09-10T18:21:52.322780+0000","last_peered":"2024-09-10T18:21:52.322780+0000","last_clean":"2024-09-10T18:21:52.322780+0000","last_became_active":"2024-09-10T18:21:44.544473+0000","last_became_peered":"2024-09-10T18:21:44.544473+0000","last_unstale":"2024-09-10T18:21:52.322780+0000","last_undegraded":"2024-09-10T18:21:52.322780+0000","last_fullsized":"2024-09-10T18:21:52.322780+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:08.848138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"59'8","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:22:55.686702+0000","last_change":"2024-09-10T18:21:46.216969+0000","last_active":"2024-09-10T18:22:55.686702+0000","last_peered":"2024-09-10T18:22:55.686702+0000","last_clean":"2024-09-10T18:22:55.686702+0000","last_became_active":"2024-09-10T18:21:46.214012+0000","last_became_peered":"2024-09-10T18:21:46.214012+0000","last_unstale":"2024-09-10T18:22:55.686702+0000","last_undegraded":"2024-09-10T18:22:55.686702+0000","last_fullsized":"2024-09-10T18:22:55.686702+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:04.460713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:53.104003+0000","last_change":"2024-09-10T18:21:48.218323+0000","last_active":"2024-09-10T18:21:53.104003+0000","last_peered":"2024-09-10T18:21:53.104003+0000","last_clean":"2024-09-10T18:21:53.104003+0000","last_became_active":"2024-09-10T18:21:48.217646+0000","last_became_peered":"2024-09-10T18:21:48.217646+0000","last_unstale":"2024-09-10T18:21:53.104003+0000","last_undegraded":"2024-09-10T18:21:53.104003+0000","last_fullsized":"2024-09-10T18:21:53.104003+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:26.421099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.004711+0000","last_change":"2024-09-10T18:21:42.189520+0000","last_active":"2024-09-10T18:21:52.004711+0000","last_peered":"2024-09-10T18:21:52.004711+0000","last_clean":"2024-09-10T18:21:52.004711+0000","last_became_active":"2024-09-10T18:21:42.189267+0000","last_became_peered":"2024-09-10T18:21:42.189267+0000","last_unstale":"2024-09-10T18:21:52.004711+0000","last_undegraded":"2024-09-10T18:21:52.004711+0000","last_fullsized":"2024-09-10T18:21:52.004711+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:58.383020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.019316+0000","last_change":"2024-09-10T18:21:44.219734+0000","last_active":"2024-09-10T18:21:52.019316+0000","last_peered":"2024-09-10T18:21:52.019316+0000","last_clean":"2024-09-10T18:21:52.019316+0000","last_became_active":"2024-09-10T18:21:44.219358+0000","last_became_peered":"2024-09-10T18:21:44.219358+0000","last_unstale":"2024-09-10T18:21:52.019316+0000","last_undegraded":"2024-09-10T18:21:52.019316+0000","last_fullsized":"2024-09-10T18:21:52.019316+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:20:50.196194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.322688+0000","last_change":"2024-09-10T18:21:46.211910+0000","last_active":"2024-09-10T18:21:52.322688+0000","last_peered":"2024-09-10T18:21:52.322688+0000","last_clean":"2024-09-10T18:21:52.322688+0000","last_became_active":"2024-09-10T18:21:46.211590+0000","last_became_peered":"2024-09-10T18:21:46.211590+0000","last_unstale":"2024-09-10T18:21:52.322688+0000","last_undegraded":"2024-09-10T18:21:52.322688+0000","last_fullsized":"2024-09-10T18:21:52.322688+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:34.185747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.006603+0000","last_change":"2024-09-10T18:21:48.247718+0000","last_active":"2024-09-10T18:21:52.006603+0000","last_peered":"2024-09-10T18:21:52.006603+0000","last_clean":"2024-09-10T18:21:52.006603+0000","last_became_active":"2024-09-10T18:21:48.247346+0000","last_became_peered":"2024-09-10T18:21:48.247346+0000","last_unstale":"2024-09-10T18:21:52.006603+0000","last_undegraded":"2024-09-10T18:21:52.006603+0000","last_fullsized":"2024-09-10T18:21:52.006603+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:02.665101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.019286+0000","last_change":"2024-09-10T18:21:42.195544+0000","last_active":"2024-09-10T18:21:52.019286+0000","last_peered":"2024-09-10T18:21:52.019286+0000","last_clean":"2024-09-10T18:21:52.019286+0000","last_became_active":"2024-09-10T18:21:42.195212+0000","last_became_peered":"2024-09-10T18:21:42.195212+0000","last_unstale":"2024-09-10T18:21:52.019286+0000","last_undegraded":"2024-09-10T18:21:52.019286+0000","last_fullsized":"2024-09-10T18:21:52.019286+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:36:46.215943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.004578+0000","last_change":"2024-09-10T18:21:44.216062+0000","last_active":"2024-09-10T18:21:52.004578+0000","last_peered":"2024-09-10T18:21:52.004578+0000","last_clean":"2024-09-10T18:21:52.004578+0000","last_became_active":"2024-09-10T18:21:44.215710+0000","last_became_peered":"2024-09-10T18:21:44.215710+0000","last_unstale":"2024-09-10T18:21:52.004578+0000","last_undegraded":"2024-09-10T18:21:52.004578+0000","last_fullsized":"2024-09-10T18:21:52.004578+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:39.612550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:53.104222+0000","last_change":"2024-09-10T18:21:46.213003+0000","last_active":"2024-09-10T18:21:53.104222+0000","last_peered":"2024-09-10T18:21:53.104222+0000","last_clean":"2024-09-10T18:21:53.104222+0000","last_became_active":"2024-09-10T18:21:46.212695+0000","last_became_peered":"2024-09-10T18:21:46.212695+0000","last_unstale":"2024-09-10T18:21:53.104222+0000","last_undegraded":"2024-09-10T18:21:53.104222+0000","last_fullsized":"2024-09-10T18:21:53.104222+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:47:09.906108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.005513+0000","last_change":"2024-09-10T18:21:48.248269+0000","last_active":"2024-09-10T18:21:52.005513+0000","last_peered":"2024-09-10T18:21:52.005513+0000","last_clean":"2024-09-10T18:21:52.005513+0000","last_became_active":"2024-09-10T18:21:48.247894+0000","last_became_peered":"2024-09-10T18:21:48.247894+0000","last_unstale":"2024-09-10T18:21:52.005513+0000","last_undegraded":"2024-09-10T18:21:52.005513+0000","last_fullsized":"2024-09-10T18:21:52.005513+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:32:58.512691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.005162+0000","last_change":"2024-09-10T18:21:42.194518+0000","last_active":"2024-09-10T18:21:52.005162+0000","last_peered":"2024-09-10T18:21:52.005162+0000","last_clean":"2024-09-10T18:21:52.005162+0000","last_became_active":"2024-09-10T18:21:42.194192+0000","last_became_peered":"2024-09-10T18:21:42.194192+0000","last_unstale":"2024-09-10T18:21:52.005162+0000","last_undegraded":"2024-09-10T18:21:52.005162+0000","last_fullsized":"2024-09-10T18:21:52.005162+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:08:08.607365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"59'12","reported_seq":39,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.005394+0000","last_change":"2024-09-10T18:21:44.214158+0000","last_active":"2024-09-10T18:21:52.005394+0000","last_peered":"2024-09-10T18:21:52.005394+0000","last_clean":"2024-09-10T18:21:52.005394+0000","last_became_active":"2024-09-10T18:21:44.213808+0000","last_became_peered":"2024-09-10T18:21:44.213808+0000","last_unstale":"2024-09-10T18:21:52.005394+0000","last_undegraded":"2024-09-10T18:21:52.005394+0000","last_fullsized":"2024-09-10T18:21:52.005394+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:59:44.036230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.006612+0000","last_change":"2024-09-10T18:21:46.214567+0000","last_active":"2024-09-10T18:21:52.006612+0000","last_peered":"2024-09-10T18:21:52.006612+0000","last_clean":"2024-09-10T18:21:52.006612+0000","last_became_active":"2024-09-10T18:21:46.214274+0000","last_became_peered":"2024-09-10T18:21:46.214274+0000","last_unstale":"2024-09-10T18:21:52.006612+0000","last_undegraded":"2024-09-10T18:21:52.006612+0000","last_fullsized":"2024-09-10T18:21:52.006612+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:35.456006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":71,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":48,"seq":206158430230,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298536,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446616,"statfs":{"total":95995035648,"available":95689334784,"internally_reserved":0,"allocated":1212416,"data_stored":823866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.57999999999999996}]},{"osd":1,"last update":"Tue Sep 10 18:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.48799999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.53200000000000003}]},{"osd":3,"last update":"Tue Sep 10 18:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.503},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.51800000000000002}]},{"osd":4,"last update":"Tue Sep 10 18:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.41299999999999998}]},{"osd":5,"last update":"Tue Sep 10 18:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.59999999999999998}]},{"osd":6,"last update":"Tue Sep 10 18:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.56100000000000005}]}]},{"osd":6,"up_from":42,"seq":180388626458,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298516,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446636,"statfs":{"total":95995035648,"available":95689355264,"internally_reserved":0,"allocated":1191936,"data_stored":822685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.59099999999999997}]},{"osd":1,"last update":"Tue Sep 10 18:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.437},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.42399999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.51900000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.38300000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.53800000000000003}]},{"osd":5,"last update":"Tue Sep 10 18:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.49399999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.47299999999999998}]}]},{"osd":5,"up_from":36,"seq":154618822687,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297936,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447216,"statfs":{"total":95995035648,"available":95689949184,"internally_reserved":0,"allocated":593920,"data_stored":231681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:22:45 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.46700000000000003,"15min":0.47499999999999998},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.58999999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.441},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51300000000000001,"15min":0.51700000000000002},"min":{"1min":0.375,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.59899999999999998,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.51800000000000002}]},{"osd":1,"last update":"Tue Sep 10 18:22:45 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.53800000000000003,"15min":0.54500000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.53000000000000003,"15min":0.53700000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.59599999999999997,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.49299999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:22:45 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.48399999999999999,"15min":0.48599999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.58199999999999996,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.44,"5min":0.50700000000000001,"15min":0.51800000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.58399999999999996,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.503}]},{"osd":3,"last update":"Tue Sep 10 18:22:45 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.51700000000000002,"15min":0.52200000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.69199999999999995,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.51800000000000002,"15min":0.52500000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.61099999999999999,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.54600000000000004}]},{"osd":4,"last update":"Tue Sep 10 18:22:45 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.40100000000000002,"15min":0.40500000000000003},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.55500000000000005,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.435,"5min":0.41999999999999998,"15min":0.41799999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.58199999999999996,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.38200000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:22:05 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.41499999999999998}]},{"osd":7,"last update":"Tue Sep 10 18:22:32 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.45000000000000001}]}]},{"osd":4,"up_from":30,"seq":128849018915,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297956,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447196,"statfs":{"total":95995035648,"available":95689928704,"internally_reserved":0,"allocated":614400,"data_stored":232113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.5,"15min":0.503},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51800000000000002,"15min":0.51900000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.81100000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.28000000000000003}]},{"osd":1,"last update":"Tue Sep 10 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.505,"15min":0.51100000000000001},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.52300000000000002,"15min":0.52500000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.32600000000000001}]},{"osd":2,"last update":"Tue Sep 10 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.50800000000000001,"15min":0.50900000000000001},"min":{"1min":0.373,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.67900000000000005,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52000000000000002,"15min":0.52200000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.33800000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54200000000000004,"15min":0.54000000000000004},"min":{"1min":0.34300000000000003,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.54700000000000004,"15min":0.55300000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.752,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.34899999999999998}]},{"osd":5,"last update":"Tue Sep 10 18:22:43 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.36499999999999999,"15min":0.35699999999999998},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.59499999999999997,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.31},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.372,"15min":0.36599999999999999},"min":{"1min":0.29299999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.623,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.29299999999999998}]},{"osd":6,"last update":"Tue Sep 10 18:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.438}]},{"osd":7,"last update":"Tue Sep 10 18:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.371},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.35999999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182441,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297976,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447176,"statfs":{"total":95995035648,"available":95689908224,"internally_reserved":0,"allocated":634880,"data_stored":232826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.38,"15min":0.38700000000000001},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.38200000000000001,"15min":0.38400000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.47399999999999998}]},{"osd":1,"last update":"Tue Sep 10 18:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.41599999999999998,"15min":0.41999999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.54600000000000004,"5min":0.56999999999999995,"15min":0.56999999999999995},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.38,"5min":0.39100000000000001,"15min":0.39300000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.51600000000000001,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.439}]},{"osd":2,"last update":"Tue Sep 10 18:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.432,"15min":0.44},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.40100000000000002,"15min":0.40699999999999997},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.54300000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.45400000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:22:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.52200000000000002,"15min":0.52400000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.54400000000000004}]},{"osd":5,"last update":"Tue Sep 10 18:22:44 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.53200000000000003,"15min":0.53800000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52400000000000002,"15min":0.52600000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.56499999999999995}]},{"osd":6,"last update":"Tue Sep 10 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.504},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.58399999999999996}]},{"osd":7,"last update":"Tue Sep 10 18:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.63300000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411374,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297944,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447208,"statfs":{"total":95995035648,"available":95689940992,"internally_reserved":0,"allocated":602112,"data_stored":232478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:22:38 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.33600000000000002,"15min":0.32200000000000001},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.52100000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.33200000000000002,"15min":0.317},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.41199999999999998}]},{"osd":1,"last update":"Tue Sep 10 18:22:38 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.33500000000000002,"15min":0.31900000000000001},"min":{"1min":0.249,"5min":0.223,"15min":0.223},"max":{"1min":0.496,"5min":0.496,"15min":0.496},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.35999999999999999,"15min":0.35399999999999998},"min":{"1min":0.252,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.51700000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.46700000000000003}]},{"osd":3,"last update":"Tue Sep 10 18:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.377,"15min":0.38},"min":{"1min":0.23699999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.51100000000000001,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.377,"5min":0.38200000000000001,"15min":0.379},"min":{"1min":0.27000000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.48599999999999999,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.497}]},{"osd":4,"last update":"Tue Sep 10 18:22:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.51900000000000002,"15min":0.51700000000000002},"min":{"1min":0.38500000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.69799999999999995}]},{"osd":5,"last update":"Tue Sep 10 18:22:43 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.53600000000000003,"15min":0.53800000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.878},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56100000000000005,"15min":0.56399999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.85699999999999998}]},{"osd":6,"last update":"Tue Sep 10 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.71799999999999997}]},{"osd":7,"last update":"Tue Sep 10 18:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.70599999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297988,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447164,"statfs":{"total":95995035648,"available":95689895936,"internally_reserved":0,"allocated":647168,"data_stored":233043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.32400000000000001,"15min":0.29699999999999999},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.32800000000000001,"15min":0.30299999999999999},"min":{"1min":0.26900000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.24099999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:22:35 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.35999999999999999,"15min":0.35699999999999998},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.437,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35199999999999998,"15min":0.34599999999999997},"min":{"1min":0.254,"5min":0.222,"15min":0.222},"max":{"1min":0.51200000000000001,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.32200000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:23:03 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.40200000000000002,"15min":0.40600000000000003},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.54000000000000004,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.38400000000000001,"15min":0.39000000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.52200000000000002,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.25700000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:22:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.54600000000000004,"15min":0.55200000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.71599999999999997,"5min":0.747,"15min":0.747},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.496,"5min":0.53600000000000003,"15min":0.54300000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.628,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.35799999999999998}]},{"osd":5,"last update":"Tue Sep 10 18:22:44 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54700000000000004,"15min":0.54900000000000004},"min":{"1min":0.40600000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.66200000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.443},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52800000000000002,"15min":0.52900000000000003},"min":{"1min":0.39300000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.70999999999999996,"5min":0.746,"15min":0.746},"last":0.33400000000000002}]},{"osd":6,"last update":"Tue Sep 10 18:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.39100000000000001}]},{"osd":7,"last update":"Tue Sep 10 18:22:29 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.38100000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738424,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298532,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446620,"statfs":{"total":95995035648,"available":95689338880,"internally_reserved":0,"allocated":1208320,"data_stored":823201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 18:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.33500000000000002,"15min":0.31900000000000001},"min":{"1min":0.27200000000000002,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.313},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.34599999999999997,"15min":0.33200000000000002},"min":{"1min":0.23899999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.54300000000000004,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.24099999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:22:38 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.39200000000000002,"15min":0.39400000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.39200000000000002,"15min":0.38400000000000001},"min":{"1min":0.26100000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.19500000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.45700000000000002,"15min":0.42499999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":4.3689999999999998,"5min":4.3689999999999998,"15min":4.3689999999999998},"last":0.251},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.435,"15min":0.42699999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.219}]},{"osd":4,"last update":"Tue Sep 10 18:22:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.50900000000000001,"15min":0.5},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.371},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51800000000000002,"15min":0.51400000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.35399999999999998}]},{"osd":5,"last update":"Tue Sep 10 18:22:42 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.40799999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.51800000000000002,"15min":0.50800000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.433}]},{"osd":6,"last update":"Tue Sep 10 18:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.36699999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.38800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T18:23:07.749 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T18:23:07.991 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:23:08.411 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:08 smithi115 ceph-mon[41073]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:08.411 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:08 smithi115 ceph-mon[41073]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:23:08.412 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:08 smithi115 ceph-mon[29603]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:08.412 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:08 smithi115 ceph-mon[29603]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:23:08.655 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:08 smithi150 ceph-mon[32761]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:08.655 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:08 smithi150 ceph-mon[32761]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:23:09.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:23:08 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:23:08] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:23:09.779 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:23:09.781 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T18:23:10.347 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:10 smithi150 ceph-mon[32761]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:10.347 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:23:10 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:23:10.422 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:10 smithi115 ceph-mon[41073]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:10.423 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:10 smithi115 ceph-mon[29603]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:10.445 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-10T18:23:09.047096+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":929,"num_read_kb":786,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":548,"ondisk_log_size":548,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385384,"kb_used_data":6548,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575832,"statfs":{"total":767960285184,"available":765517651968,"internally_reserved":0,"allocated":6705152,"data_stored":3631893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003024"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.004738+0000","last_change":"2024-09-10T18:21:48.249635+0000","last_active":"2024-09-10T18:21:52.004738+0000","last_peered":"2024-09-10T18:21:52.004738+0000","last_clean":"2024-09-10T18:21:52.004738+0000","last_became_active":"2024-09-10T18:21:48.249190+0000","last_became_peered":"2024-09-10T18:21:48.249190+0000","last_unstale":"2024-09-10T18:21:52.004738+0000","last_undegraded":"2024-09-10T18:21:52.004738+0000","last_fullsized":"2024-09-10T18:21:52.004738+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:27:20.808410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.322935+0000","last_change":"2024-09-10T18:21:42.193527+0000","last_active":"2024-09-10T18:21:52.322935+0000","last_peered":"2024-09-10T18:21:52.322935+0000","last_clean":"2024-09-10T18:21:52.322935+0000","last_became_active":"2024-09-10T18:21:42.193245+0000","last_became_peered":"2024-09-10T18:21:42.193245+0000","last_unstale":"2024-09-10T18:21:52.322935+0000","last_undegraded":"2024-09-10T18:21:52.322935+0000","last_fullsized":"2024-09-10T18:21:52.322935+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:05:31.863462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.004796+0000","last_change":"2024-09-10T18:21:44.214718+0000","last_active":"2024-09-10T18:21:52.004796+0000","last_peered":"2024-09-10T18:21:52.004796+0000","last_clean":"2024-09-10T18:21:52.004796+0000","last_became_active":"2024-09-10T18:21:44.214441+0000","last_became_peered":"2024-09-10T18:21:44.214441+0000","last_unstale":"2024-09-10T18:21:52.004796+0000","last_undegraded":"2024-09-10T18:21:52.004796+0000","last_fullsized":"2024-09-10T18:21:52.004796+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:56.693341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.006028+0000","last_change":"2024-09-10T18:21:46.217211+0000","last_active":"2024-09-10T18:21:52.006028+0000","last_peered":"2024-09-10T18:21:52.006028+0000","last_clean":"2024-09-10T18:21:52.006028+0000","last_became_active":"2024-09-10T18:21:46.216944+0000","last_became_peered":"2024-09-10T18:21:46.216944+0000","last_unstale":"2024-09-10T18:21:52.006028+0000","last_undegraded":"2024-09-10T18:21:52.006028+0000","last_fullsized":"2024-09-10T18:21:52.006028+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:49.462516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.005614+0000","last_change":"2024-09-10T18:21:42.193333+0000","last_active":"2024-09-10T18:21:52.005614+0000","last_peered":"2024-09-10T18:21:52.005614+0000","last_clean":"2024-09-10T18:21:52.005614+0000","last_became_active":"2024-09-10T18:21:42.193080+0000","last_became_peered":"2024-09-10T18:21:42.193080+0000","last_unstale":"2024-09-10T18:21:52.005614+0000","last_undegraded":"2024-09-10T18:21:52.005614+0000","last_fullsized":"2024-09-10T18:21:52.005614+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:56:14.627342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.006366+0000","last_change":"2024-09-10T18:21:44.213781+0000","last_active":"2024-09-10T18:21:52.006366+0000","last_peered":"2024-09-10T18:21:52.006366+0000","last_clean":"2024-09-10T18:21:52.006366+0000","last_became_active":"2024-09-10T18:21:44.213506+0000","last_became_peered":"2024-09-10T18:21:44.213506+0000","last_unstale":"2024-09-10T18:21:52.006366+0000","last_undegraded":"2024-09-10T18:21:52.006366+0000","last_fullsized":"2024-09-10T18:21:52.006366+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:55:23.286343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:53.103332+0000","last_change":"2024-09-10T18:21:46.211107+0000","last_active":"2024-09-10T18:21:53.103332+0000","last_peered":"2024-09-10T18:21:53.103332+0000","last_clean":"2024-09-10T18:21:53.103332+0000","last_became_active":"2024-09-10T18:21:46.210710+0000","last_became_peered":"2024-09-10T18:21:46.210710+0000","last_unstale":"2024-09-10T18:21:53.103332+0000","last_undegraded":"2024-09-10T18:21:53.103332+0000","last_fullsized":"2024-09-10T18:21:53.103332+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:38:58.779887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.006059+0000","last_change":"2024-09-10T18:21:48.215402+0000","last_active":"2024-09-10T18:21:52.006059+0000","last_peered":"2024-09-10T18:21:52.006059+0000","last_clean":"2024-09-10T18:21:52.006059+0000","last_became_active":"2024-09-10T18:21:48.215147+0000","last_became_peered":"2024-09-10T18:21:48.215147+0000","last_unstale":"2024-09-10T18:21:52.006059+0000","last_undegraded":"2024-09-10T18:21:52.006059+0000","last_fullsized":"2024-09-10T18:21:52.006059+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:51.469720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"52'1","reported_seq":27,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.397414+0000","last_change":"2024-09-10T18:21:42.196527+0000","last_active":"2024-09-10T18:21:52.397414+0000","last_peered":"2024-09-10T18:21:52.397414+0000","last_clean":"2024-09-10T18:21:52.397414+0000","last_became_active":"2024-09-10T18:21:42.196243+0000","last_became_peered":"2024-09-10T18:21:42.196243+0000","last_unstale":"2024-09-10T18:21:52.397414+0000","last_undegraded":"2024-09-10T18:21:52.397414+0000","last_fullsized":"2024-09-10T18:21:52.397414+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:28.682300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"59'5","reported_seq":26,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.006227+0000","last_change":"2024-09-10T18:21:44.218505+0000","last_active":"2024-09-10T18:21:52.006227+0000","last_peered":"2024-09-10T18:21:52.006227+0000","last_clean":"2024-09-10T18:21:52.006227+0000","last_became_active":"2024-09-10T18:21:44.218210+0000","last_became_peered":"2024-09-10T18:21:44.218210+0000","last_unstale":"2024-09-10T18:21:52.006227+0000","last_undegraded":"2024-09-10T18:21:52.006227+0000","last_fullsized":"2024-09-10T18:21:52.006227+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:16.477789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.396922+0000","last_change":"2024-09-10T18:21:46.216369+0000","last_active":"2024-09-10T18:21:52.396922+0000","last_peered":"2024-09-10T18:21:52.396922+0000","last_clean":"2024-09-10T18:21:52.396922+0000","last_became_active":"2024-09-10T18:21:46.216087+0000","last_became_peered":"2024-09-10T18:21:46.216087+0000","last_unstale":"2024-09-10T18:21:52.396922+0000","last_undegraded":"2024-09-10T18:21:52.396922+0000","last_fullsized":"2024-09-10T18:21:52.396922+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:40.439486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.382555+0000","last_change":"2024-09-10T18:21:48.220939+0000","last_active":"2024-09-10T18:21:52.382555+0000","last_peered":"2024-09-10T18:21:52.382555+0000","last_clean":"2024-09-10T18:21:52.382555+0000","last_became_active":"2024-09-10T18:21:48.220645+0000","last_became_peered":"2024-09-10T18:21:48.220645+0000","last_unstale":"2024-09-10T18:21:52.382555+0000","last_undegraded":"2024-09-10T18:21:52.382555+0000","last_fullsized":"2024-09-10T18:21:52.382555+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:58.021282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.397263+0000","last_change":"2024-09-10T18:21:42.192801+0000","last_active":"2024-09-10T18:21:52.397263+0000","last_peered":"2024-09-10T18:21:52.397263+0000","last_clean":"2024-09-10T18:21:52.397263+0000","last_became_active":"2024-09-10T18:21:42.192299+0000","last_became_peered":"2024-09-10T18:21:42.192299+0000","last_unstale":"2024-09-10T18:21:52.397263+0000","last_undegraded":"2024-09-10T18:21:52.397263+0000","last_fullsized":"2024-09-10T18:21:52.397263+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:29:09.257584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.006281+0000","last_change":"2024-09-10T18:21:44.212626+0000","last_active":"2024-09-10T18:21:52.006281+0000","last_peered":"2024-09-10T18:21:52.006281+0000","last_clean":"2024-09-10T18:21:52.006281+0000","last_became_active":"2024-09-10T18:21:44.211888+0000","last_became_peered":"2024-09-10T18:21:44.211888+0000","last_unstale":"2024-09-10T18:21:52.006281+0000","last_undegraded":"2024-09-10T18:21:52.006281+0000","last_fullsized":"2024-09-10T18:21:52.006281+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:56.119082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.382747+0000","last_change":"2024-09-10T18:21:46.223065+0000","last_active":"2024-09-10T18:21:52.382747+0000","last_peered":"2024-09-10T18:21:52.382747+0000","last_clean":"2024-09-10T18:21:52.382747+0000","last_became_active":"2024-09-10T18:21:46.222792+0000","last_became_peered":"2024-09-10T18:21:46.222792+0000","last_unstale":"2024-09-10T18:21:52.382747+0000","last_undegraded":"2024-09-10T18:21:52.382747+0000","last_fullsized":"2024-09-10T18:21:52.382747+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:47.666989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.322005+0000","last_change":"2024-09-10T18:21:48.224521+0000","last_active":"2024-09-10T18:21:52.322005+0000","last_peered":"2024-09-10T18:21:52.322005+0000","last_clean":"2024-09-10T18:21:52.322005+0000","last_became_active":"2024-09-10T18:21:48.224241+0000","last_became_peered":"2024-09-10T18:21:48.224241+0000","last_unstale":"2024-09-10T18:21:52.322005+0000","last_undegraded":"2024-09-10T18:21:52.322005+0000","last_fullsized":"2024-09-10T18:21:52.322005+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:45:52.473703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"59'17","reported_seq":44,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.006688+0000","last_change":"2024-09-10T18:21:44.213864+0000","last_active":"2024-09-10T18:21:52.006688+0000","last_peered":"2024-09-10T18:21:52.006688+0000","last_clean":"2024-09-10T18:21:52.006688+0000","last_became_active":"2024-09-10T18:21:44.213420+0000","last_became_peered":"2024-09-10T18:21:44.213420+0000","last_unstale":"2024-09-10T18:21:52.006688+0000","last_undegraded":"2024-09-10T18:21:52.006688+0000","last_fullsized":"2024-09-10T18:21:52.006688+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:39.523394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.397448+0000","last_change":"2024-09-10T18:21:42.191952+0000","last_active":"2024-09-10T18:21:52.397448+0000","last_peered":"2024-09-10T18:21:52.397448+0000","last_clean":"2024-09-10T18:21:52.397448+0000","last_became_active":"2024-09-10T18:21:42.191546+0000","last_became_peered":"2024-09-10T18:21:42.191546+0000","last_unstale":"2024-09-10T18:21:52.397448+0000","last_undegraded":"2024-09-10T18:21:52.397448+0000","last_fullsized":"2024-09-10T18:21:52.397448+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:40.565015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:53.103985+0000","last_change":"2024-09-10T18:21:46.213690+0000","last_active":"2024-09-10T18:21:53.103985+0000","last_peered":"2024-09-10T18:21:53.103985+0000","last_clean":"2024-09-10T18:21:53.103985+0000","last_became_active":"2024-09-10T18:21:46.213385+0000","last_became_peered":"2024-09-10T18:21:46.213385+0000","last_unstale":"2024-09-10T18:21:53.103985+0000","last_undegraded":"2024-09-10T18:21:53.103985+0000","last_fullsized":"2024-09-10T18:21:53.103985+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:36:09.241715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.004382+0000","last_change":"2024-09-10T18:21:48.222236+0000","last_active":"2024-09-10T18:21:52.004382+0000","last_peered":"2024-09-10T18:21:52.004382+0000","last_clean":"2024-09-10T18:21:52.004382+0000","last_became_active":"2024-09-10T18:21:48.221959+0000","last_became_peered":"2024-09-10T18:21:48.221959+0000","last_unstale":"2024-09-10T18:21:52.004382+0000","last_undegraded":"2024-09-10T18:21:52.004382+0000","last_fullsized":"2024-09-10T18:21:52.004382+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:58.843453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.006528+0000","last_change":"2024-09-10T18:21:44.212233+0000","last_active":"2024-09-10T18:21:52.006528+0000","last_peered":"2024-09-10T18:21:52.006528+0000","last_clean":"2024-09-10T18:21:52.006528+0000","last_became_active":"2024-09-10T18:21:44.211440+0000","last_became_peered":"2024-09-10T18:21:44.211440+0000","last_unstale":"2024-09-10T18:21:52.006528+0000","last_undegraded":"2024-09-10T18:21:52.006528+0000","last_fullsized":"2024-09-10T18:21:52.006528+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:32.793112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:53.103764+0000","last_change":"2024-09-10T18:21:42.189586+0000","last_active":"2024-09-10T18:21:53.103764+0000","last_peered":"2024-09-10T18:21:53.103764+0000","last_clean":"2024-09-10T18:21:53.103764+0000","last_became_active":"2024-09-10T18:21:42.189177+0000","last_became_peered":"2024-09-10T18:21:42.189177+0000","last_unstale":"2024-09-10T18:21:53.103764+0000","last_undegraded":"2024-09-10T18:21:53.103764+0000","last_fullsized":"2024-09-10T18:21:53.103764+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:16:41.693895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"59'8","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:22:55.686544+0000","last_change":"2024-09-10T18:21:46.217678+0000","last_active":"2024-09-10T18:22:55.686544+0000","last_peered":"2024-09-10T18:22:55.686544+0000","last_clean":"2024-09-10T18:22:55.686544+0000","last_became_active":"2024-09-10T18:21:46.217352+0000","last_became_peered":"2024-09-10T18:21:46.217352+0000","last_unstale":"2024-09-10T18:22:55.686544+0000","last_undegraded":"2024-09-10T18:22:55.686544+0000","last_fullsized":"2024-09-10T18:22:55.686544+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:10:35.296112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.006522+0000","last_change":"2024-09-10T18:21:48.217792+0000","last_active":"2024-09-10T18:21:52.006522+0000","last_peered":"2024-09-10T18:21:52.006522+0000","last_clean":"2024-09-10T18:21:52.006522+0000","last_became_active":"2024-09-10T18:21:48.217514+0000","last_became_peered":"2024-09-10T18:21:48.217514+0000","last_unstale":"2024-09-10T18:21:52.006522+0000","last_undegraded":"2024-09-10T18:21:52.006522+0000","last_fullsized":"2024-09-10T18:21:52.006522+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:00:40.116274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:53.103078+0000","last_change":"2024-09-10T18:21:44.213153+0000","last_active":"2024-09-10T18:21:53.103078+0000","last_peered":"2024-09-10T18:21:53.103078+0000","last_clean":"2024-09-10T18:21:53.103078+0000","last_became_active":"2024-09-10T18:21:44.212834+0000","last_became_peered":"2024-09-10T18:21:44.212834+0000","last_unstale":"2024-09-10T18:21:53.103078+0000","last_undegraded":"2024-09-10T18:21:53.103078+0000","last_fullsized":"2024-09-10T18:21:53.103078+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:17:58.381424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:53.103713+0000","last_change":"2024-09-10T18:21:42.188114+0000","last_active":"2024-09-10T18:21:53.103713+0000","last_peered":"2024-09-10T18:21:53.103713+0000","last_clean":"2024-09-10T18:21:53.103713+0000","last_became_active":"2024-09-10T18:21:42.187866+0000","last_became_peered":"2024-09-10T18:21:42.187866+0000","last_unstale":"2024-09-10T18:21:53.103713+0000","last_undegraded":"2024-09-10T18:21:53.103713+0000","last_fullsized":"2024-09-10T18:21:53.103713+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:57.842295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:22:55.686802+0000","last_change":"2024-09-10T18:21:46.223557+0000","last_active":"2024-09-10T18:22:55.686802+0000","last_peered":"2024-09-10T18:22:55.686802+0000","last_clean":"2024-09-10T18:22:55.686802+0000","last_became_active":"2024-09-10T18:21:46.223238+0000","last_became_peered":"2024-09-10T18:21:46.223238+0000","last_unstale":"2024-09-10T18:22:55.686802+0000","last_undegraded":"2024-09-10T18:22:55.686802+0000","last_fullsized":"2024-09-10T18:22:55.686802+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:14:38.870392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.383003+0000","last_change":"2024-09-10T18:21:48.219291+0000","last_active":"2024-09-10T18:21:52.383003+0000","last_peered":"2024-09-10T18:21:52.383003+0000","last_clean":"2024-09-10T18:21:52.383003+0000","last_became_active":"2024-09-10T18:21:48.218997+0000","last_became_peered":"2024-09-10T18:21:48.218997+0000","last_unstale":"2024-09-10T18:21:52.383003+0000","last_undegraded":"2024-09-10T18:21:52.383003+0000","last_fullsized":"2024-09-10T18:21:52.383003+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:37.892083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.006681+0000","last_change":"2024-09-10T18:21:44.212496+0000","last_active":"2024-09-10T18:21:52.006681+0000","last_peered":"2024-09-10T18:21:52.006681+0000","last_clean":"2024-09-10T18:21:52.006681+0000","last_became_active":"2024-09-10T18:21:44.212172+0000","last_became_peered":"2024-09-10T18:21:44.212172+0000","last_unstale":"2024-09-10T18:21:52.006681+0000","last_undegraded":"2024-09-10T18:21:52.006681+0000","last_fullsized":"2024-09-10T18:21:52.006681+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:58:12.578630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.396931+0000","last_change":"2024-09-10T18:21:42.197518+0000","last_active":"2024-09-10T18:21:52.396931+0000","last_peered":"2024-09-10T18:21:52.396931+0000","last_clean":"2024-09-10T18:21:52.396931+0000","last_became_active":"2024-09-10T18:21:42.197278+0000","last_became_peered":"2024-09-10T18:21:42.197278+0000","last_unstale":"2024-09-10T18:21:52.396931+0000","last_undegraded":"2024-09-10T18:21:52.396931+0000","last_fullsized":"2024-09-10T18:21:52.396931+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:12:57.036608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.383182+0000","last_change":"2024-09-10T18:21:46.222424+0000","last_active":"2024-09-10T18:21:52.383182+0000","last_peered":"2024-09-10T18:21:52.383182+0000","last_clean":"2024-09-10T18:21:52.383182+0000","last_became_active":"2024-09-10T18:21:46.221946+0000","last_became_peered":"2024-09-10T18:21:46.221946+0000","last_unstale":"2024-09-10T18:21:52.383182+0000","last_undegraded":"2024-09-10T18:21:52.383182+0000","last_fullsized":"2024-09-10T18:21:52.383182+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:06.041139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.004977+0000","last_change":"2024-09-10T18:21:48.248206+0000","last_active":"2024-09-10T18:21:52.004977+0000","last_peered":"2024-09-10T18:21:52.004977+0000","last_clean":"2024-09-10T18:21:52.004977+0000","last_became_active":"2024-09-10T18:21:48.247899+0000","last_became_peered":"2024-09-10T18:21:48.247899+0000","last_unstale":"2024-09-10T18:21:52.004977+0000","last_undegraded":"2024-09-10T18:21:52.004977+0000","last_fullsized":"2024-09-10T18:21:52.004977+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:48.371158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"59'14","reported_seq":37,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.006107+0000","last_change":"2024-09-10T18:21:44.211671+0000","last_active":"2024-09-10T18:21:52.006107+0000","last_peered":"2024-09-10T18:21:52.006107+0000","last_clean":"2024-09-10T18:21:52.006107+0000","last_became_active":"2024-09-10T18:21:44.211372+0000","last_became_peered":"2024-09-10T18:21:44.211372+0000","last_unstale":"2024-09-10T18:21:52.006107+0000","last_undegraded":"2024-09-10T18:21:52.006107+0000","last_fullsized":"2024-09-10T18:21:52.006107+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:11:54.539741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.019384+0000","last_change":"2024-09-10T18:21:42.195666+0000","last_active":"2024-09-10T18:21:52.019384+0000","last_peered":"2024-09-10T18:21:52.019384+0000","last_clean":"2024-09-10T18:21:52.019384+0000","last_became_active":"2024-09-10T18:21:42.195270+0000","last_became_peered":"2024-09-10T18:21:42.195270+0000","last_unstale":"2024-09-10T18:21:52.019384+0000","last_undegraded":"2024-09-10T18:21:52.019384+0000","last_fullsized":"2024-09-10T18:21:52.019384+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:21.188035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:22:55.686344+0000","last_change":"2024-09-10T18:21:46.219050+0000","last_active":"2024-09-10T18:22:55.686344+0000","last_peered":"2024-09-10T18:22:55.686344+0000","last_clean":"2024-09-10T18:22:55.686344+0000","last_became_active":"2024-09-10T18:21:46.218699+0000","last_became_peered":"2024-09-10T18:21:46.218699+0000","last_unstale":"2024-09-10T18:22:55.686344+0000","last_undegraded":"2024-09-10T18:22:55.686344+0000","last_fullsized":"2024-09-10T18:22:55.686344+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:56.293720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.397302+0000","last_change":"2024-09-10T18:21:48.249560+0000","last_active":"2024-09-10T18:21:52.397302+0000","last_peered":"2024-09-10T18:21:52.397302+0000","last_clean":"2024-09-10T18:21:52.397302+0000","last_became_active":"2024-09-10T18:21:48.249246+0000","last_became_peered":"2024-09-10T18:21:48.249246+0000","last_unstale":"2024-09-10T18:21:52.397302+0000","last_undegraded":"2024-09-10T18:21:52.397302+0000","last_fullsized":"2024-09-10T18:21:52.397302+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:15.844229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"59'18","reported_seq":48,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.004403+0000","last_change":"2024-09-10T18:21:44.544806+0000","last_active":"2024-09-10T18:21:52.004403+0000","last_peered":"2024-09-10T18:21:52.004403+0000","last_clean":"2024-09-10T18:21:52.004403+0000","last_became_active":"2024-09-10T18:21:44.544490+0000","last_became_peered":"2024-09-10T18:21:44.544490+0000","last_unstale":"2024-09-10T18:21:52.004403+0000","last_undegraded":"2024-09-10T18:21:52.004403+0000","last_fullsized":"2024-09-10T18:21:52.004403+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:27.004328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:53.104434+0000","last_change":"2024-09-10T18:21:42.189983+0000","last_active":"2024-09-10T18:21:53.104434+0000","last_peered":"2024-09-10T18:21:53.104434+0000","last_clean":"2024-09-10T18:21:53.104434+0000","last_became_active":"2024-09-10T18:21:42.189725+0000","last_became_peered":"2024-09-10T18:21:42.189725+0000","last_unstale":"2024-09-10T18:21:53.104434+0000","last_undegraded":"2024-09-10T18:21:53.104434+0000","last_fullsized":"2024-09-10T18:21:53.104434+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:21:23.864069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.006028+0000","last_change":"2024-09-10T18:21:46.215416+0000","last_active":"2024-09-10T18:21:52.006028+0000","last_peered":"2024-09-10T18:21:52.006028+0000","last_clean":"2024-09-10T18:21:52.006028+0000","last_became_active":"2024-09-10T18:21:46.215158+0000","last_became_peered":"2024-09-10T18:21:46.215158+0000","last_unstale":"2024-09-10T18:21:52.006028+0000","last_undegraded":"2024-09-10T18:21:52.006028+0000","last_fullsized":"2024-09-10T18:21:52.006028+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:58.888442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.006281+0000","last_change":"2024-09-10T18:21:48.220047+0000","last_active":"2024-09-10T18:21:52.006281+0000","last_peered":"2024-09-10T18:21:52.006281+0000","last_clean":"2024-09-10T18:21:52.006281+0000","last_became_active":"2024-09-10T18:21:48.219746+0000","last_became_peered":"2024-09-10T18:21:48.219746+0000","last_unstale":"2024-09-10T18:21:52.006281+0000","last_undegraded":"2024-09-10T18:21:52.006281+0000","last_fullsized":"2024-09-10T18:21:52.006281+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:05.530770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"59'19","reported_seq":52,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.322610+0000","last_change":"2024-09-10T18:21:44.544653+0000","last_active":"2024-09-10T18:21:52.322610+0000","last_peered":"2024-09-10T18:21:52.322610+0000","last_clean":"2024-09-10T18:21:52.322610+0000","last_became_active":"2024-09-10T18:21:44.544384+0000","last_became_peered":"2024-09-10T18:21:44.544384+0000","last_unstale":"2024-09-10T18:21:52.322610+0000","last_undegraded":"2024-09-10T18:21:52.322610+0000","last_fullsized":"2024-09-10T18:21:52.322610+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:46:34.914351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.397398+0000","last_change":"2024-09-10T18:21:42.197195+0000","last_active":"2024-09-10T18:21:52.397398+0000","last_peered":"2024-09-10T18:21:52.397398+0000","last_clean":"2024-09-10T18:21:52.397398+0000","last_became_active":"2024-09-10T18:21:42.196932+0000","last_became_peered":"2024-09-10T18:21:42.196932+0000","last_unstale":"2024-09-10T18:21:52.397398+0000","last_undegraded":"2024-09-10T18:21:52.397398+0000","last_fullsized":"2024-09-10T18:21:52.397398+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:40.088144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.019269+0000","last_change":"2024-09-10T18:21:46.217608+0000","last_active":"2024-09-10T18:21:52.019269+0000","last_peered":"2024-09-10T18:21:52.019269+0000","last_clean":"2024-09-10T18:21:52.019269+0000","last_became_active":"2024-09-10T18:21:46.217319+0000","last_became_peered":"2024-09-10T18:21:46.217319+0000","last_unstale":"2024-09-10T18:21:52.019269+0000","last_undegraded":"2024-09-10T18:21:52.019269+0000","last_fullsized":"2024-09-10T18:21:52.019269+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:22.721887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:53.103234+0000","last_change":"2024-09-10T18:21:48.248170+0000","last_active":"2024-09-10T18:21:53.103234+0000","last_peered":"2024-09-10T18:21:53.103234+0000","last_clean":"2024-09-10T18:21:53.103234+0000","last_became_active":"2024-09-10T18:21:48.247879+0000","last_became_peered":"2024-09-10T18:21:48.247879+0000","last_unstale":"2024-09-10T18:21:53.103234+0000","last_undegraded":"2024-09-10T18:21:53.103234+0000","last_fullsized":"2024-09-10T18:21:53.103234+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:16:47.722052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:53.102811+0000","last_change":"2024-09-10T18:21:44.215899+0000","last_active":"2024-09-10T18:21:53.102811+0000","last_peered":"2024-09-10T18:21:53.102811+0000","last_clean":"2024-09-10T18:21:53.102811+0000","last_became_active":"2024-09-10T18:21:44.214567+0000","last_became_peered":"2024-09-10T18:21:44.214567+0000","last_unstale":"2024-09-10T18:21:53.102811+0000","last_undegraded":"2024-09-10T18:21:53.102811+0000","last_fullsized":"2024-09-10T18:21:53.102811+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:51.040990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:53.104012+0000","last_change":"2024-09-10T18:21:42.187509+0000","last_active":"2024-09-10T18:21:53.104012+0000","last_peered":"2024-09-10T18:21:53.104012+0000","last_clean":"2024-09-10T18:21:53.104012+0000","last_became_active":"2024-09-10T18:21:42.187265+0000","last_became_peered":"2024-09-10T18:21:42.187265+0000","last_unstale":"2024-09-10T18:21:53.104012+0000","last_undegraded":"2024-09-10T18:21:53.104012+0000","last_fullsized":"2024-09-10T18:21:53.104012+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:35:55.312030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:22:55.686336+0000","last_change":"2024-09-10T18:21:46.216058+0000","last_active":"2024-09-10T18:22:55.686336+0000","last_peered":"2024-09-10T18:22:55.686336+0000","last_clean":"2024-09-10T18:22:55.686336+0000","last_became_active":"2024-09-10T18:21:46.215783+0000","last_became_peered":"2024-09-10T18:21:46.215783+0000","last_unstale":"2024-09-10T18:22:55.686336+0000","last_undegraded":"2024-09-10T18:22:55.686336+0000","last_fullsized":"2024-09-10T18:22:55.686336+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:06.900935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.321885+0000","last_change":"2024-09-10T18:21:48.223222+0000","last_active":"2024-09-10T18:21:52.321885+0000","last_peered":"2024-09-10T18:21:52.321885+0000","last_clean":"2024-09-10T18:21:52.321885+0000","last_became_active":"2024-09-10T18:21:48.222967+0000","last_became_peered":"2024-09-10T18:21:48.222967+0000","last_unstale":"2024-09-10T18:21:52.321885+0000","last_undegraded":"2024-09-10T18:21:52.321885+0000","last_fullsized":"2024-09-10T18:21:52.321885+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:26:53.598813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"59'28","reported_seq":80,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:22:55.686344+0000","last_change":"2024-09-10T18:21:44.219913+0000","last_active":"2024-09-10T18:22:55.686344+0000","last_peered":"2024-09-10T18:22:55.686344+0000","last_clean":"2024-09-10T18:22:55.686344+0000","last_became_active":"2024-09-10T18:21:44.219627+0000","last_became_peered":"2024-09-10T18:21:44.219627+0000","last_unstale":"2024-09-10T18:22:55.686344+0000","last_undegraded":"2024-09-10T18:22:55.686344+0000","last_fullsized":"2024-09-10T18:22:55.686344+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:23:21.885343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.382779+0000","last_change":"2024-09-10T18:21:42.192716+0000","last_active":"2024-09-10T18:21:52.382779+0000","last_peered":"2024-09-10T18:21:52.382779+0000","last_clean":"2024-09-10T18:21:52.382779+0000","last_became_active":"2024-09-10T18:21:42.192352+0000","last_became_peered":"2024-09-10T18:21:42.192352+0000","last_unstale":"2024-09-10T18:21:52.382779+0000","last_undegraded":"2024-09-10T18:21:52.382779+0000","last_fullsized":"2024-09-10T18:21:52.382779+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:29:59.794433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.322576+0000","last_change":"2024-09-10T18:21:46.213595+0000","last_active":"2024-09-10T18:21:52.322576+0000","last_peered":"2024-09-10T18:21:52.322576+0000","last_clean":"2024-09-10T18:21:52.322576+0000","last_became_active":"2024-09-10T18:21:46.212638+0000","last_became_peered":"2024-09-10T18:21:46.212638+0000","last_unstale":"2024-09-10T18:21:52.322576+0000","last_undegraded":"2024-09-10T18:21:52.322576+0000","last_fullsized":"2024-09-10T18:21:52.322576+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:35:49.916818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.005472+0000","last_change":"2024-09-10T18:21:48.222223+0000","last_active":"2024-09-10T18:21:52.005472+0000","last_peered":"2024-09-10T18:21:52.005472+0000","last_clean":"2024-09-10T18:21:52.005472+0000","last_became_active":"2024-09-10T18:21:48.221981+0000","last_became_peered":"2024-09-10T18:21:48.221981+0000","last_unstale":"2024-09-10T18:21:52.005472+0000","last_undegraded":"2024-09-10T18:21:52.005472+0000","last_fullsized":"2024-09-10T18:21:52.005472+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:45:36.100976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"59'13","reported_seq":43,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:53.104535+0000","last_change":"2024-09-10T18:21:44.216116+0000","last_active":"2024-09-10T18:21:53.104535+0000","last_peered":"2024-09-10T18:21:53.104535+0000","last_clean":"2024-09-10T18:21:53.104535+0000","last_became_active":"2024-09-10T18:21:44.215282+0000","last_became_peered":"2024-09-10T18:21:44.215282+0000","last_unstale":"2024-09-10T18:21:53.104535+0000","last_undegraded":"2024-09-10T18:21:53.104535+0000","last_fullsized":"2024-09-10T18:21:53.104535+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:26:55.462477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"60'5","reported_seq":102,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:23:03.819109+0000","last_change":"2024-09-10T18:21:46.203518+0000","last_active":"2024-09-10T18:23:03.819109+0000","last_peered":"2024-09-10T18:23:03.819109+0000","last_clean":"2024-09-10T18:23:03.819109+0000","last_became_active":"2024-09-10T18:21:42.190312+0000","last_became_peered":"2024-09-10T18:21:42.190312+0000","last_unstale":"2024-09-10T18:23:03.819109+0000","last_undegraded":"2024-09-10T18:23:03.819109+0000","last_fullsized":"2024-09-10T18:23:03.819109+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:09:41.138079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051195600000000004,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":76,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.003802+0000","last_change":"2024-09-10T18:21:42.188339+0000","last_active":"2024-09-10T18:21:52.003802+0000","last_peered":"2024-09-10T18:21:52.003802+0000","last_clean":"2024-09-10T18:21:52.003802+0000","last_became_active":"2024-09-10T18:21:42.188062+0000","last_became_peered":"2024-09-10T18:21:42.188062+0000","last_unstale":"2024-09-10T18:21:52.003802+0000","last_undegraded":"2024-09-10T18:21:52.003802+0000","last_fullsized":"2024-09-10T18:21:52.003802+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:22:00.353358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.004133+0000","last_change":"2024-09-10T18:21:46.220317+0000","last_active":"2024-09-10T18:21:52.004133+0000","last_peered":"2024-09-10T18:21:52.004133+0000","last_clean":"2024-09-10T18:21:52.004133+0000","last_became_active":"2024-09-10T18:21:46.220039+0000","last_became_peered":"2024-09-10T18:21:46.220039+0000","last_unstale":"2024-09-10T18:21:52.004133+0000","last_undegraded":"2024-09-10T18:21:52.004133+0000","last_fullsized":"2024-09-10T18:21:52.004133+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:35:18.190468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.397147+0000","last_change":"2024-09-10T18:21:48.249852+0000","last_active":"2024-09-10T18:21:52.397147+0000","last_peered":"2024-09-10T18:21:52.397147+0000","last_clean":"2024-09-10T18:21:52.397147+0000","last_became_active":"2024-09-10T18:21:48.248690+0000","last_became_peered":"2024-09-10T18:21:48.248690+0000","last_unstale":"2024-09-10T18:21:52.397147+0000","last_undegraded":"2024-09-10T18:21:52.397147+0000","last_fullsized":"2024-09-10T18:21:52.397147+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:26:28.559200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"59'12","reported_seq":34,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.321822+0000","last_change":"2024-09-10T18:21:44.218401+0000","last_active":"2024-09-10T18:21:52.321822+0000","last_peered":"2024-09-10T18:21:52.321822+0000","last_clean":"2024-09-10T18:21:52.321822+0000","last_became_active":"2024-09-10T18:21:44.218026+0000","last_became_peered":"2024-09-10T18:21:44.218026+0000","last_unstale":"2024-09-10T18:21:52.321822+0000","last_undegraded":"2024-09-10T18:21:52.321822+0000","last_fullsized":"2024-09-10T18:21:52.321822+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:53.692887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"54'1","reported_seq":24,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.322233+0000","last_change":"2024-09-10T18:21:46.204896+0000","last_active":"2024-09-10T18:21:52.322233+0000","last_peered":"2024-09-10T18:21:52.322233+0000","last_clean":"2024-09-10T18:21:52.322233+0000","last_became_active":"2024-09-10T18:21:42.193041+0000","last_became_peered":"2024-09-10T18:21:42.193041+0000","last_unstale":"2024-09-10T18:21:52.322233+0000","last_undegraded":"2024-09-10T18:21:52.322233+0000","last_fullsized":"2024-09-10T18:21:52.322233+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:38.003652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098752999999999992,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.396985+0000","last_change":"2024-09-10T18:21:42.197813+0000","last_active":"2024-09-10T18:21:52.396985+0000","last_peered":"2024-09-10T18:21:52.396985+0000","last_clean":"2024-09-10T18:21:52.396985+0000","last_became_active":"2024-09-10T18:21:42.197499+0000","last_became_peered":"2024-09-10T18:21:42.197499+0000","last_unstale":"2024-09-10T18:21:52.396985+0000","last_undegraded":"2024-09-10T18:21:52.396985+0000","last_fullsized":"2024-09-10T18:21:52.396985+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:32.117794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.383218+0000","last_change":"2024-09-10T18:21:46.218067+0000","last_active":"2024-09-10T18:21:52.383218+0000","last_peered":"2024-09-10T18:21:52.383218+0000","last_clean":"2024-09-10T18:21:52.383218+0000","last_became_active":"2024-09-10T18:21:46.214073+0000","last_became_peered":"2024-09-10T18:21:46.214073+0000","last_unstale":"2024-09-10T18:21:52.383218+0000","last_undegraded":"2024-09-10T18:21:52.383218+0000","last_fullsized":"2024-09-10T18:21:52.383218+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:57:48.766455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:53.103896+0000","last_change":"2024-09-10T18:21:48.221395+0000","last_active":"2024-09-10T18:21:53.103896+0000","last_peered":"2024-09-10T18:21:53.103896+0000","last_clean":"2024-09-10T18:21:53.103896+0000","last_became_active":"2024-09-10T18:21:48.221130+0000","last_became_peered":"2024-09-10T18:21:48.221130+0000","last_unstale":"2024-09-10T18:21:53.103896+0000","last_undegraded":"2024-09-10T18:21:53.103896+0000","last_fullsized":"2024-09-10T18:21:53.103896+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:21.546806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"59'14","reported_seq":38,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.019313+0000","last_change":"2024-09-10T18:21:44.545097+0000","last_active":"2024-09-10T18:21:52.019313+0000","last_peered":"2024-09-10T18:21:52.019313+0000","last_clean":"2024-09-10T18:21:52.019313+0000","last_became_active":"2024-09-10T18:21:44.544597+0000","last_became_peered":"2024-09-10T18:21:44.544597+0000","last_unstale":"2024-09-10T18:21:52.019313+0000","last_undegraded":"2024-09-10T18:21:52.019313+0000","last_fullsized":"2024-09-10T18:21:52.019313+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:54:28.205142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'2","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.005251+0000","last_change":"2024-09-10T18:21:46.209528+0000","last_active":"2024-09-10T18:21:52.005251+0000","last_peered":"2024-09-10T18:21:52.005251+0000","last_clean":"2024-09-10T18:21:52.005251+0000","last_became_active":"2024-09-10T18:21:42.190380+0000","last_became_peered":"2024-09-10T18:21:42.190380+0000","last_unstale":"2024-09-10T18:21:52.005251+0000","last_undegraded":"2024-09-10T18:21:52.005251+0000","last_fullsized":"2024-09-10T18:21:52.005251+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:15:02.719844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094658199999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.382880+0000","last_change":"2024-09-10T18:21:42.195366+0000","last_active":"2024-09-10T18:21:52.382880+0000","last_peered":"2024-09-10T18:21:52.382880+0000","last_clean":"2024-09-10T18:21:52.382880+0000","last_became_active":"2024-09-10T18:21:42.195095+0000","last_became_peered":"2024-09-10T18:21:42.195095+0000","last_unstale":"2024-09-10T18:21:52.382880+0000","last_undegraded":"2024-09-10T18:21:52.382880+0000","last_fullsized":"2024-09-10T18:21:52.382880+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:24:18.917978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"61'79","reported_seq":52,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:22:21.090419+0000","last_change":"2024-09-10T18:21:31.484955+0000","last_active":"2024-09-10T18:22:21.090419+0000","last_peered":"2024-09-10T18:22:21.090419+0000","last_clean":"2024-09-10T18:22:21.090419+0000","last_became_active":"2024-09-10T18:21:31.477446+0000","last_became_peered":"2024-09-10T18:21:31.477446+0000","last_unstale":"2024-09-10T18:22:21.090419+0000","last_undegraded":"2024-09-10T18:22:21.090419+0000","last_fullsized":"2024-09-10T18:22:21.090419+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:19:32.548162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:19:32.548162+0000","last_clean_scrub_stamp":"2024-09-10T18:19:32.548162+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:04.128475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.397055+0000","last_change":"2024-09-10T18:21:46.218393+0000","last_active":"2024-09-10T18:21:52.397055+0000","last_peered":"2024-09-10T18:21:52.397055+0000","last_clean":"2024-09-10T18:21:52.397055+0000","last_became_active":"2024-09-10T18:21:46.218107+0000","last_became_peered":"2024-09-10T18:21:46.218107+0000","last_unstale":"2024-09-10T18:21:52.397055+0000","last_undegraded":"2024-09-10T18:21:52.397055+0000","last_fullsized":"2024-09-10T18:21:52.397055+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:36:11.032542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.383250+0000","last_change":"2024-09-10T18:21:48.225584+0000","last_active":"2024-09-10T18:21:52.383250+0000","last_peered":"2024-09-10T18:21:52.383250+0000","last_clean":"2024-09-10T18:21:52.383250+0000","last_became_active":"2024-09-10T18:21:48.225234+0000","last_became_peered":"2024-09-10T18:21:48.225234+0000","last_unstale":"2024-09-10T18:21:52.383250+0000","last_undegraded":"2024-09-10T18:21:52.383250+0000","last_fullsized":"2024-09-10T18:21:52.383250+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:31.131019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"59'19","reported_seq":47,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.019160+0000","last_change":"2024-09-10T18:21:44.544859+0000","last_active":"2024-09-10T18:21:52.019160+0000","last_peered":"2024-09-10T18:21:52.019160+0000","last_clean":"2024-09-10T18:21:52.019160+0000","last_became_active":"2024-09-10T18:21:44.544505+0000","last_became_peered":"2024-09-10T18:21:44.544505+0000","last_unstale":"2024-09-10T18:21:52.019160+0000","last_undegraded":"2024-09-10T18:21:52.019160+0000","last_fullsized":"2024-09-10T18:21:52.019160+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:35:28.420372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.004782+0000","last_change":"2024-09-10T18:21:42.190925+0000","last_active":"2024-09-10T18:21:52.004782+0000","last_peered":"2024-09-10T18:21:52.004782+0000","last_clean":"2024-09-10T18:21:52.004782+0000","last_became_active":"2024-09-10T18:21:42.190660+0000","last_became_peered":"2024-09-10T18:21:42.190660+0000","last_unstale":"2024-09-10T18:21:52.004782+0000","last_undegraded":"2024-09-10T18:21:52.004782+0000","last_fullsized":"2024-09-10T18:21:52.004782+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:24:24.760672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.004046+0000","last_change":"2024-09-10T18:21:46.218021+0000","last_active":"2024-09-10T18:21:52.004046+0000","last_peered":"2024-09-10T18:21:52.004046+0000","last_clean":"2024-09-10T18:21:52.004046+0000","last_became_active":"2024-09-10T18:21:46.217382+0000","last_became_peered":"2024-09-10T18:21:46.217382+0000","last_unstale":"2024-09-10T18:21:52.004046+0000","last_undegraded":"2024-09-10T18:21:52.004046+0000","last_fullsized":"2024-09-10T18:21:52.004046+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:27:17.458917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.397042+0000","last_change":"2024-09-10T18:21:48.225762+0000","last_active":"2024-09-10T18:21:52.397042+0000","last_peered":"2024-09-10T18:21:52.397042+0000","last_clean":"2024-09-10T18:21:52.397042+0000","last_became_active":"2024-09-10T18:21:48.225273+0000","last_became_peered":"2024-09-10T18:21:48.225273+0000","last_unstale":"2024-09-10T18:21:52.397042+0000","last_undegraded":"2024-09-10T18:21:52.397042+0000","last_fullsized":"2024-09-10T18:21:52.397042+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:00.622136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.321816+0000","last_change":"2024-09-10T18:21:44.219194+0000","last_active":"2024-09-10T18:21:52.321816+0000","last_peered":"2024-09-10T18:21:52.321816+0000","last_clean":"2024-09-10T18:21:52.321816+0000","last_became_active":"2024-09-10T18:21:44.218208+0000","last_became_peered":"2024-09-10T18:21:44.218208+0000","last_unstale":"2024-09-10T18:21:52.321816+0000","last_undegraded":"2024-09-10T18:21:52.321816+0000","last_fullsized":"2024-09-10T18:21:52.321816+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:23.592157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:53.104335+0000","last_change":"2024-09-10T18:21:42.186979+0000","last_active":"2024-09-10T18:21:53.104335+0000","last_peered":"2024-09-10T18:21:53.104335+0000","last_clean":"2024-09-10T18:21:53.104335+0000","last_became_active":"2024-09-10T18:21:42.186715+0000","last_became_peered":"2024-09-10T18:21:42.186715+0000","last_unstale":"2024-09-10T18:21:53.104335+0000","last_undegraded":"2024-09-10T18:21:53.104335+0000","last_fullsized":"2024-09-10T18:21:53.104335+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:16:27.222177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.003898+0000","last_change":"2024-09-10T18:21:46.213893+0000","last_active":"2024-09-10T18:21:52.003898+0000","last_peered":"2024-09-10T18:21:52.003898+0000","last_clean":"2024-09-10T18:21:52.003898+0000","last_became_active":"2024-09-10T18:21:46.213635+0000","last_became_peered":"2024-09-10T18:21:46.213635+0000","last_unstale":"2024-09-10T18:21:52.003898+0000","last_undegraded":"2024-09-10T18:21:52.003898+0000","last_fullsized":"2024-09-10T18:21:52.003898+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:10:35.774133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.322083+0000","last_change":"2024-09-10T18:21:48.222057+0000","last_active":"2024-09-10T18:21:52.322083+0000","last_peered":"2024-09-10T18:21:52.322083+0000","last_clean":"2024-09-10T18:21:52.322083+0000","last_became_active":"2024-09-10T18:21:48.221787+0000","last_became_peered":"2024-09-10T18:21:48.221787+0000","last_unstale":"2024-09-10T18:21:52.322083+0000","last_undegraded":"2024-09-10T18:21:52.322083+0000","last_fullsized":"2024-09-10T18:21:52.322083+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:10.803275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.382496+0000","last_change":"2024-09-10T18:21:44.543735+0000","last_active":"2024-09-10T18:21:52.382496+0000","last_peered":"2024-09-10T18:21:52.382496+0000","last_clean":"2024-09-10T18:21:52.382496+0000","last_became_active":"2024-09-10T18:21:44.543366+0000","last_became_peered":"2024-09-10T18:21:44.543366+0000","last_unstale":"2024-09-10T18:21:52.382496+0000","last_undegraded":"2024-09-10T18:21:52.382496+0000","last_fullsized":"2024-09-10T18:21:52.382496+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:12:02.537666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"52'1","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.004348+0000","last_change":"2024-09-10T18:21:42.192027+0000","last_active":"2024-09-10T18:21:52.004348+0000","last_peered":"2024-09-10T18:21:52.004348+0000","last_clean":"2024-09-10T18:21:52.004348+0000","last_became_active":"2024-09-10T18:21:42.191768+0000","last_became_peered":"2024-09-10T18:21:42.191768+0000","last_unstale":"2024-09-10T18:21:52.004348+0000","last_undegraded":"2024-09-10T18:21:52.004348+0000","last_fullsized":"2024-09-10T18:21:52.004348+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:00.027576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:22:55.686702+0000","last_change":"2024-09-10T18:21:46.219325+0000","last_active":"2024-09-10T18:22:55.686702+0000","last_peered":"2024-09-10T18:22:55.686702+0000","last_clean":"2024-09-10T18:22:55.686702+0000","last_became_active":"2024-09-10T18:21:46.218717+0000","last_became_peered":"2024-09-10T18:21:46.218717+0000","last_unstale":"2024-09-10T18:22:55.686702+0000","last_undegraded":"2024-09-10T18:22:55.686702+0000","last_fullsized":"2024-09-10T18:22:55.686702+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:49:23.946749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.382615+0000","last_change":"2024-09-10T18:21:48.249077+0000","last_active":"2024-09-10T18:21:52.382615+0000","last_peered":"2024-09-10T18:21:52.382615+0000","last_clean":"2024-09-10T18:21:52.382615+0000","last_became_active":"2024-09-10T18:21:48.248808+0000","last_became_peered":"2024-09-10T18:21:48.248808+0000","last_unstale":"2024-09-10T18:21:52.382615+0000","last_undegraded":"2024-09-10T18:21:52.382615+0000","last_fullsized":"2024-09-10T18:21:52.382615+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:00:31.037243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"59'12","reported_seq":39,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.006053+0000","last_change":"2024-09-10T18:21:44.212445+0000","last_active":"2024-09-10T18:21:52.006053+0000","last_peered":"2024-09-10T18:21:52.006053+0000","last_clean":"2024-09-10T18:21:52.006053+0000","last_became_active":"2024-09-10T18:21:44.212098+0000","last_became_peered":"2024-09-10T18:21:44.212098+0000","last_unstale":"2024-09-10T18:21:52.006053+0000","last_undegraded":"2024-09-10T18:21:52.006053+0000","last_fullsized":"2024-09-10T18:21:52.006053+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:37:40.320627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"52'2","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.006738+0000","last_change":"2024-09-10T18:21:42.187924+0000","last_active":"2024-09-10T18:21:52.006738+0000","last_peered":"2024-09-10T18:21:52.006738+0000","last_clean":"2024-09-10T18:21:52.006738+0000","last_became_active":"2024-09-10T18:21:42.187638+0000","last_became_peered":"2024-09-10T18:21:42.187638+0000","last_unstale":"2024-09-10T18:21:52.006738+0000","last_undegraded":"2024-09-10T18:21:52.006738+0000","last_fullsized":"2024-09-10T18:21:52.006738+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:48:21.137550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.003675+0000","last_change":"2024-09-10T18:21:46.213109+0000","last_active":"2024-09-10T18:21:52.003675+0000","last_peered":"2024-09-10T18:21:52.003675+0000","last_clean":"2024-09-10T18:21:52.003675+0000","last_became_active":"2024-09-10T18:21:46.212841+0000","last_became_peered":"2024-09-10T18:21:46.212841+0000","last_unstale":"2024-09-10T18:21:52.003675+0000","last_undegraded":"2024-09-10T18:21:52.003675+0000","last_fullsized":"2024-09-10T18:21:52.003675+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:35:20.777465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.004970+0000","last_change":"2024-09-10T18:21:48.221562+0000","last_active":"2024-09-10T18:21:52.004970+0000","last_peered":"2024-09-10T18:21:52.004970+0000","last_clean":"2024-09-10T18:21:52.004970+0000","last_became_active":"2024-09-10T18:21:48.221314+0000","last_became_peered":"2024-09-10T18:21:48.221314+0000","last_unstale":"2024-09-10T18:21:52.004970+0000","last_undegraded":"2024-09-10T18:21:52.004970+0000","last_fullsized":"2024-09-10T18:21:52.004970+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:27:34.121104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.322819+0000","last_change":"2024-09-10T18:21:44.543784+0000","last_active":"2024-09-10T18:21:52.322819+0000","last_peered":"2024-09-10T18:21:52.322819+0000","last_clean":"2024-09-10T18:21:52.322819+0000","last_became_active":"2024-09-10T18:21:44.543498+0000","last_became_peered":"2024-09-10T18:21:44.543498+0000","last_unstale":"2024-09-10T18:21:52.322819+0000","last_undegraded":"2024-09-10T18:21:52.322819+0000","last_fullsized":"2024-09-10T18:21:52.322819+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:40:42.061165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.003798+0000","last_change":"2024-09-10T18:21:42.186885+0000","last_active":"2024-09-10T18:21:52.003798+0000","last_peered":"2024-09-10T18:21:52.003798+0000","last_clean":"2024-09-10T18:21:52.003798+0000","last_became_active":"2024-09-10T18:21:42.186604+0000","last_became_peered":"2024-09-10T18:21:42.186604+0000","last_unstale":"2024-09-10T18:21:52.003798+0000","last_undegraded":"2024-09-10T18:21:52.003798+0000","last_fullsized":"2024-09-10T18:21:52.003798+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:05:28.202909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.005598+0000","last_change":"2024-09-10T18:21:46.218222+0000","last_active":"2024-09-10T18:21:52.005598+0000","last_peered":"2024-09-10T18:21:52.005598+0000","last_clean":"2024-09-10T18:21:52.005598+0000","last_became_active":"2024-09-10T18:21:46.217986+0000","last_became_peered":"2024-09-10T18:21:46.217986+0000","last_unstale":"2024-09-10T18:21:52.005598+0000","last_undegraded":"2024-09-10T18:21:52.005598+0000","last_fullsized":"2024-09-10T18:21:52.005598+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:05:27.223317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.004322+0000","last_change":"2024-09-10T18:21:48.216201+0000","last_active":"2024-09-10T18:21:52.004322+0000","last_peered":"2024-09-10T18:21:52.004322+0000","last_clean":"2024-09-10T18:21:52.004322+0000","last_became_active":"2024-09-10T18:21:48.215951+0000","last_became_peered":"2024-09-10T18:21:48.215951+0000","last_unstale":"2024-09-10T18:21:52.004322+0000","last_undegraded":"2024-09-10T18:21:52.004322+0000","last_fullsized":"2024-09-10T18:21:52.004322+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:49:46.528068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.019019+0000","last_change":"2024-09-10T18:21:42.193990+0000","last_active":"2024-09-10T18:21:52.019019+0000","last_peered":"2024-09-10T18:21:52.019019+0000","last_clean":"2024-09-10T18:21:52.019019+0000","last_became_active":"2024-09-10T18:21:42.193479+0000","last_became_peered":"2024-09-10T18:21:42.193479+0000","last_unstale":"2024-09-10T18:21:52.019019+0000","last_undegraded":"2024-09-10T18:21:52.019019+0000","last_fullsized":"2024-09-10T18:21:52.019019+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:17.563667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"59'6","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.005842+0000","last_change":"2024-09-10T18:21:44.544793+0000","last_active":"2024-09-10T18:21:52.005842+0000","last_peered":"2024-09-10T18:21:52.005842+0000","last_clean":"2024-09-10T18:21:52.005842+0000","last_became_active":"2024-09-10T18:21:44.544457+0000","last_became_peered":"2024-09-10T18:21:44.544457+0000","last_unstale":"2024-09-10T18:21:52.005842+0000","last_undegraded":"2024-09-10T18:21:52.005842+0000","last_fullsized":"2024-09-10T18:21:52.005842+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:40:09.752979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.383141+0000","last_change":"2024-09-10T18:21:46.215678+0000","last_active":"2024-09-10T18:21:52.383141+0000","last_peered":"2024-09-10T18:21:52.383141+0000","last_clean":"2024-09-10T18:21:52.383141+0000","last_became_active":"2024-09-10T18:21:46.215365+0000","last_became_peered":"2024-09-10T18:21:46.215365+0000","last_unstale":"2024-09-10T18:21:52.383141+0000","last_undegraded":"2024-09-10T18:21:52.383141+0000","last_fullsized":"2024-09-10T18:21:52.383141+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:05.489735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.397137+0000","last_change":"2024-09-10T18:21:48.249927+0000","last_active":"2024-09-10T18:21:52.397137+0000","last_peered":"2024-09-10T18:21:52.397137+0000","last_clean":"2024-09-10T18:21:52.397137+0000","last_became_active":"2024-09-10T18:21:48.249663+0000","last_became_peered":"2024-09-10T18:21:48.249663+0000","last_unstale":"2024-09-10T18:21:52.397137+0000","last_undegraded":"2024-09-10T18:21:52.397137+0000","last_fullsized":"2024-09-10T18:21:52.397137+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:33.485123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.005245+0000","last_change":"2024-09-10T18:21:44.543239+0000","last_active":"2024-09-10T18:21:52.005245+0000","last_peered":"2024-09-10T18:21:52.005245+0000","last_clean":"2024-09-10T18:21:52.005245+0000","last_became_active":"2024-09-10T18:21:44.542923+0000","last_became_peered":"2024-09-10T18:21:44.542923+0000","last_unstale":"2024-09-10T18:21:52.005245+0000","last_undegraded":"2024-09-10T18:21:52.005245+0000","last_fullsized":"2024-09-10T18:21:52.005245+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:03:40.909083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.382781+0000","last_change":"2024-09-10T18:21:42.193818+0000","last_active":"2024-09-10T18:21:52.382781+0000","last_peered":"2024-09-10T18:21:52.382781+0000","last_clean":"2024-09-10T18:21:52.382781+0000","last_became_active":"2024-09-10T18:21:42.193532+0000","last_became_peered":"2024-09-10T18:21:42.193532+0000","last_unstale":"2024-09-10T18:21:52.382781+0000","last_undegraded":"2024-09-10T18:21:52.382781+0000","last_fullsized":"2024-09-10T18:21:52.382781+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:03:04.235353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:22:55.686769+0000","last_change":"2024-09-10T18:21:46.215394+0000","last_active":"2024-09-10T18:22:55.686769+0000","last_peered":"2024-09-10T18:22:55.686769+0000","last_clean":"2024-09-10T18:22:55.686769+0000","last_became_active":"2024-09-10T18:21:46.215063+0000","last_became_peered":"2024-09-10T18:21:46.215063+0000","last_unstale":"2024-09-10T18:22:55.686769+0000","last_undegraded":"2024-09-10T18:22:55.686769+0000","last_fullsized":"2024-09-10T18:22:55.686769+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:45:37.828828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.322586+0000","last_change":"2024-09-10T18:21:48.223251+0000","last_active":"2024-09-10T18:21:52.322586+0000","last_peered":"2024-09-10T18:21:52.322586+0000","last_clean":"2024-09-10T18:21:52.322586+0000","last_became_active":"2024-09-10T18:21:48.222995+0000","last_became_peered":"2024-09-10T18:21:48.222995+0000","last_unstale":"2024-09-10T18:21:52.322586+0000","last_undegraded":"2024-09-10T18:21:52.322586+0000","last_fullsized":"2024-09-10T18:21:52.322586+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:09:02.531330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.383097+0000","last_change":"2024-09-10T18:21:44.544958+0000","last_active":"2024-09-10T18:21:52.383097+0000","last_peered":"2024-09-10T18:21:52.383097+0000","last_clean":"2024-09-10T18:21:52.383097+0000","last_became_active":"2024-09-10T18:21:44.544654+0000","last_became_peered":"2024-09-10T18:21:44.544654+0000","last_unstale":"2024-09-10T18:21:52.383097+0000","last_undegraded":"2024-09-10T18:21:52.383097+0000","last_fullsized":"2024-09-10T18:21:52.383097+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:56:46.447734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.322781+0000","last_change":"2024-09-10T18:21:42.195177+0000","last_active":"2024-09-10T18:21:52.322781+0000","last_peered":"2024-09-10T18:21:52.322781+0000","last_clean":"2024-09-10T18:21:52.322781+0000","last_became_active":"2024-09-10T18:21:42.194408+0000","last_became_peered":"2024-09-10T18:21:42.194408+0000","last_unstale":"2024-09-10T18:21:52.322781+0000","last_undegraded":"2024-09-10T18:21:52.322781+0000","last_fullsized":"2024-09-10T18:21:52.322781+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:36.889116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:22:55.686405+0000","last_change":"2024-09-10T18:21:46.222267+0000","last_active":"2024-09-10T18:22:55.686405+0000","last_peered":"2024-09-10T18:22:55.686405+0000","last_clean":"2024-09-10T18:22:55.686405+0000","last_became_active":"2024-09-10T18:21:46.221973+0000","last_became_peered":"2024-09-10T18:21:46.221973+0000","last_unstale":"2024-09-10T18:22:55.686405+0000","last_undegraded":"2024-09-10T18:22:55.686405+0000","last_fullsized":"2024-09-10T18:22:55.686405+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:18:19.797607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.006565+0000","last_change":"2024-09-10T18:21:48.216075+0000","last_active":"2024-09-10T18:21:52.006565+0000","last_peered":"2024-09-10T18:21:52.006565+0000","last_clean":"2024-09-10T18:21:52.006565+0000","last_became_active":"2024-09-10T18:21:48.215762+0000","last_became_peered":"2024-09-10T18:21:48.215762+0000","last_unstale":"2024-09-10T18:21:52.006565+0000","last_undegraded":"2024-09-10T18:21:52.006565+0000","last_fullsized":"2024-09-10T18:21:52.006565+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:33.770854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:53.102812+0000","last_change":"2024-09-10T18:21:44.216152+0000","last_active":"2024-09-10T18:21:53.102812+0000","last_peered":"2024-09-10T18:21:53.102812+0000","last_clean":"2024-09-10T18:21:53.102812+0000","last_became_active":"2024-09-10T18:21:44.215371+0000","last_became_peered":"2024-09-10T18:21:44.215371+0000","last_unstale":"2024-09-10T18:21:53.102812+0000","last_undegraded":"2024-09-10T18:21:53.102812+0000","last_fullsized":"2024-09-10T18:21:53.102812+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:13.942356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.019180+0000","last_change":"2024-09-10T18:21:42.193635+0000","last_active":"2024-09-10T18:21:52.019180+0000","last_peered":"2024-09-10T18:21:52.019180+0000","last_clean":"2024-09-10T18:21:52.019180+0000","last_became_active":"2024-09-10T18:21:42.193319+0000","last_became_peered":"2024-09-10T18:21:42.193319+0000","last_unstale":"2024-09-10T18:21:52.019180+0000","last_undegraded":"2024-09-10T18:21:52.019180+0000","last_fullsized":"2024-09-10T18:21:52.019180+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:41:03.510218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.005050+0000","last_change":"2024-09-10T18:21:46.219503+0000","last_active":"2024-09-10T18:21:52.005050+0000","last_peered":"2024-09-10T18:21:52.005050+0000","last_clean":"2024-09-10T18:21:52.005050+0000","last_became_active":"2024-09-10T18:21:46.219222+0000","last_became_peered":"2024-09-10T18:21:46.219222+0000","last_unstale":"2024-09-10T18:21:52.005050+0000","last_undegraded":"2024-09-10T18:21:52.005050+0000","last_fullsized":"2024-09-10T18:21:52.005050+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:25.256348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.321860+0000","last_change":"2024-09-10T18:21:48.225120+0000","last_active":"2024-09-10T18:21:52.321860+0000","last_peered":"2024-09-10T18:21:52.321860+0000","last_clean":"2024-09-10T18:21:52.321860+0000","last_became_active":"2024-09-10T18:21:48.224860+0000","last_became_peered":"2024-09-10T18:21:48.224860+0000","last_unstale":"2024-09-10T18:21:52.321860+0000","last_undegraded":"2024-09-10T18:21:52.321860+0000","last_fullsized":"2024-09-10T18:21:52.321860+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:44:12.980814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.006312+0000","last_change":"2024-09-10T18:21:44.213806+0000","last_active":"2024-09-10T18:21:52.006312+0000","last_peered":"2024-09-10T18:21:52.006312+0000","last_clean":"2024-09-10T18:21:52.006312+0000","last_became_active":"2024-09-10T18:21:44.213496+0000","last_became_peered":"2024-09-10T18:21:44.213496+0000","last_unstale":"2024-09-10T18:21:52.006312+0000","last_undegraded":"2024-09-10T18:21:52.006312+0000","last_fullsized":"2024-09-10T18:21:52.006312+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:52.815568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:53.104340+0000","last_change":"2024-09-10T18:21:42.189078+0000","last_active":"2024-09-10T18:21:53.104340+0000","last_peered":"2024-09-10T18:21:53.104340+0000","last_clean":"2024-09-10T18:21:53.104340+0000","last_became_active":"2024-09-10T18:21:42.188150+0000","last_became_peered":"2024-09-10T18:21:42.188150+0000","last_unstale":"2024-09-10T18:21:53.104340+0000","last_undegraded":"2024-09-10T18:21:53.104340+0000","last_fullsized":"2024-09-10T18:21:53.104340+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:17:05.659061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:53.103629+0000","last_change":"2024-09-10T18:21:46.216514+0000","last_active":"2024-09-10T18:21:53.103629+0000","last_peered":"2024-09-10T18:21:53.103629+0000","last_clean":"2024-09-10T18:21:53.103629+0000","last_became_active":"2024-09-10T18:21:46.215932+0000","last_became_peered":"2024-09-10T18:21:46.215932+0000","last_unstale":"2024-09-10T18:21:53.103629+0000","last_undegraded":"2024-09-10T18:21:53.103629+0000","last_fullsized":"2024-09-10T18:21:53.103629+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:08:13.255160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.004276+0000","last_change":"2024-09-10T18:21:48.224686+0000","last_active":"2024-09-10T18:21:52.004276+0000","last_peered":"2024-09-10T18:21:52.004276+0000","last_clean":"2024-09-10T18:21:52.004276+0000","last_became_active":"2024-09-10T18:21:48.224423+0000","last_became_peered":"2024-09-10T18:21:48.224423+0000","last_unstale":"2024-09-10T18:21:52.004276+0000","last_undegraded":"2024-09-10T18:21:52.004276+0000","last_fullsized":"2024-09-10T18:21:52.004276+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:16.684266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"59'4","reported_seq":22,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.004317+0000","last_change":"2024-09-10T18:21:44.218991+0000","last_active":"2024-09-10T18:21:52.004317+0000","last_peered":"2024-09-10T18:21:52.004317+0000","last_clean":"2024-09-10T18:21:52.004317+0000","last_became_active":"2024-09-10T18:21:44.218658+0000","last_became_peered":"2024-09-10T18:21:44.218658+0000","last_unstale":"2024-09-10T18:21:52.004317+0000","last_undegraded":"2024-09-10T18:21:52.004317+0000","last_fullsized":"2024-09-10T18:21:52.004317+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:39:28.896129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.383201+0000","last_change":"2024-09-10T18:21:42.193238+0000","last_active":"2024-09-10T18:21:52.383201+0000","last_peered":"2024-09-10T18:21:52.383201+0000","last_clean":"2024-09-10T18:21:52.383201+0000","last_became_active":"2024-09-10T18:21:42.192955+0000","last_became_peered":"2024-09-10T18:21:42.192955+0000","last_unstale":"2024-09-10T18:21:52.383201+0000","last_undegraded":"2024-09-10T18:21:52.383201+0000","last_fullsized":"2024-09-10T18:21:52.383201+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:00:41.711996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.018874+0000","last_change":"2024-09-10T18:21:46.214983+0000","last_active":"2024-09-10T18:21:52.018874+0000","last_peered":"2024-09-10T18:21:52.018874+0000","last_clean":"2024-09-10T18:21:52.018874+0000","last_became_active":"2024-09-10T18:21:46.214542+0000","last_became_peered":"2024-09-10T18:21:46.214542+0000","last_unstale":"2024-09-10T18:21:52.018874+0000","last_undegraded":"2024-09-10T18:21:52.018874+0000","last_fullsized":"2024-09-10T18:21:52.018874+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:44.015035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.396896+0000","last_change":"2024-09-10T18:21:48.216479+0000","last_active":"2024-09-10T18:21:52.396896+0000","last_peered":"2024-09-10T18:21:52.396896+0000","last_clean":"2024-09-10T18:21:52.396896+0000","last_became_active":"2024-09-10T18:21:48.216204+0000","last_became_peered":"2024-09-10T18:21:48.216204+0000","last_unstale":"2024-09-10T18:21:52.396896+0000","last_undegraded":"2024-09-10T18:21:52.396896+0000","last_fullsized":"2024-09-10T18:21:52.396896+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:17:30.545376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.004576+0000","last_change":"2024-09-10T18:21:44.544622+0000","last_active":"2024-09-10T18:21:52.004576+0000","last_peered":"2024-09-10T18:21:52.004576+0000","last_clean":"2024-09-10T18:21:52.004576+0000","last_became_active":"2024-09-10T18:21:44.544384+0000","last_became_peered":"2024-09-10T18:21:44.544384+0000","last_unstale":"2024-09-10T18:21:52.004576+0000","last_undegraded":"2024-09-10T18:21:52.004576+0000","last_fullsized":"2024-09-10T18:21:52.004576+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:57:03.121884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.019367+0000","last_change":"2024-09-10T18:21:42.191810+0000","last_active":"2024-09-10T18:21:52.019367+0000","last_peered":"2024-09-10T18:21:52.019367+0000","last_clean":"2024-09-10T18:21:52.019367+0000","last_became_active":"2024-09-10T18:21:42.191560+0000","last_became_peered":"2024-09-10T18:21:42.191560+0000","last_unstale":"2024-09-10T18:21:52.019367+0000","last_undegraded":"2024-09-10T18:21:52.019367+0000","last_fullsized":"2024-09-10T18:21:52.019367+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:35:20.356744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.396782+0000","last_change":"2024-09-10T18:21:46.217609+0000","last_active":"2024-09-10T18:21:52.396782+0000","last_peered":"2024-09-10T18:21:52.396782+0000","last_clean":"2024-09-10T18:21:52.396782+0000","last_became_active":"2024-09-10T18:21:46.217313+0000","last_became_peered":"2024-09-10T18:21:46.217313+0000","last_unstale":"2024-09-10T18:21:52.396782+0000","last_undegraded":"2024-09-10T18:21:52.396782+0000","last_fullsized":"2024-09-10T18:21:52.396782+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:31:45.563334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.004839+0000","last_change":"2024-09-10T18:21:48.249570+0000","last_active":"2024-09-10T18:21:52.004839+0000","last_peered":"2024-09-10T18:21:52.004839+0000","last_clean":"2024-09-10T18:21:52.004839+0000","last_became_active":"2024-09-10T18:21:48.249257+0000","last_became_peered":"2024-09-10T18:21:48.249257+0000","last_unstale":"2024-09-10T18:21:52.004839+0000","last_undegraded":"2024-09-10T18:21:52.004839+0000","last_fullsized":"2024-09-10T18:21:52.004839+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:55.391437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.397041+0000","last_change":"2024-09-10T18:21:48.215302+0000","last_active":"2024-09-10T18:21:52.397041+0000","last_peered":"2024-09-10T18:21:52.397041+0000","last_clean":"2024-09-10T18:21:52.397041+0000","last_became_active":"2024-09-10T18:21:48.214978+0000","last_became_peered":"2024-09-10T18:21:48.214978+0000","last_unstale":"2024-09-10T18:21:52.397041+0000","last_undegraded":"2024-09-10T18:21:52.397041+0000","last_fullsized":"2024-09-10T18:21:52.397041+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:57:22.002791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.382568+0000","last_change":"2024-09-10T18:21:42.192531+0000","last_active":"2024-09-10T18:21:52.382568+0000","last_peered":"2024-09-10T18:21:52.382568+0000","last_clean":"2024-09-10T18:21:52.382568+0000","last_became_active":"2024-09-10T18:21:42.190963+0000","last_became_peered":"2024-09-10T18:21:42.190963+0000","last_unstale":"2024-09-10T18:21:52.382568+0000","last_undegraded":"2024-09-10T18:21:52.382568+0000","last_fullsized":"2024-09-10T18:21:52.382568+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:30:14.112856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.322780+0000","last_change":"2024-09-10T18:21:44.544740+0000","last_active":"2024-09-10T18:21:52.322780+0000","last_peered":"2024-09-10T18:21:52.322780+0000","last_clean":"2024-09-10T18:21:52.322780+0000","last_became_active":"2024-09-10T18:21:44.544473+0000","last_became_peered":"2024-09-10T18:21:44.544473+0000","last_unstale":"2024-09-10T18:21:52.322780+0000","last_undegraded":"2024-09-10T18:21:52.322780+0000","last_fullsized":"2024-09-10T18:21:52.322780+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:08.848138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"59'8","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:22:55.686702+0000","last_change":"2024-09-10T18:21:46.216969+0000","last_active":"2024-09-10T18:22:55.686702+0000","last_peered":"2024-09-10T18:22:55.686702+0000","last_clean":"2024-09-10T18:22:55.686702+0000","last_became_active":"2024-09-10T18:21:46.214012+0000","last_became_peered":"2024-09-10T18:21:46.214012+0000","last_unstale":"2024-09-10T18:22:55.686702+0000","last_undegraded":"2024-09-10T18:22:55.686702+0000","last_fullsized":"2024-09-10T18:22:55.686702+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:04.460713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:53.104003+0000","last_change":"2024-09-10T18:21:48.218323+0000","last_active":"2024-09-10T18:21:53.104003+0000","last_peered":"2024-09-10T18:21:53.104003+0000","last_clean":"2024-09-10T18:21:53.104003+0000","last_became_active":"2024-09-10T18:21:48.217646+0000","last_became_peered":"2024-09-10T18:21:48.217646+0000","last_unstale":"2024-09-10T18:21:53.104003+0000","last_undegraded":"2024-09-10T18:21:53.104003+0000","last_fullsized":"2024-09-10T18:21:53.104003+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:26.421099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.004711+0000","last_change":"2024-09-10T18:21:42.189520+0000","last_active":"2024-09-10T18:21:52.004711+0000","last_peered":"2024-09-10T18:21:52.004711+0000","last_clean":"2024-09-10T18:21:52.004711+0000","last_became_active":"2024-09-10T18:21:42.189267+0000","last_became_peered":"2024-09-10T18:21:42.189267+0000","last_unstale":"2024-09-10T18:21:52.004711+0000","last_undegraded":"2024-09-10T18:21:52.004711+0000","last_fullsized":"2024-09-10T18:21:52.004711+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:58.383020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.019316+0000","last_change":"2024-09-10T18:21:44.219734+0000","last_active":"2024-09-10T18:21:52.019316+0000","last_peered":"2024-09-10T18:21:52.019316+0000","last_clean":"2024-09-10T18:21:52.019316+0000","last_became_active":"2024-09-10T18:21:44.219358+0000","last_became_peered":"2024-09-10T18:21:44.219358+0000","last_unstale":"2024-09-10T18:21:52.019316+0000","last_undegraded":"2024-09-10T18:21:52.019316+0000","last_fullsized":"2024-09-10T18:21:52.019316+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:20:50.196194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.322688+0000","last_change":"2024-09-10T18:21:46.211910+0000","last_active":"2024-09-10T18:21:52.322688+0000","last_peered":"2024-09-10T18:21:52.322688+0000","last_clean":"2024-09-10T18:21:52.322688+0000","last_became_active":"2024-09-10T18:21:46.211590+0000","last_became_peered":"2024-09-10T18:21:46.211590+0000","last_unstale":"2024-09-10T18:21:52.322688+0000","last_undegraded":"2024-09-10T18:21:52.322688+0000","last_fullsized":"2024-09-10T18:21:52.322688+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:34.185747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.006603+0000","last_change":"2024-09-10T18:21:48.247718+0000","last_active":"2024-09-10T18:21:52.006603+0000","last_peered":"2024-09-10T18:21:52.006603+0000","last_clean":"2024-09-10T18:21:52.006603+0000","last_became_active":"2024-09-10T18:21:48.247346+0000","last_became_peered":"2024-09-10T18:21:48.247346+0000","last_unstale":"2024-09-10T18:21:52.006603+0000","last_undegraded":"2024-09-10T18:21:52.006603+0000","last_fullsized":"2024-09-10T18:21:52.006603+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:02.665101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.019286+0000","last_change":"2024-09-10T18:21:42.195544+0000","last_active":"2024-09-10T18:21:52.019286+0000","last_peered":"2024-09-10T18:21:52.019286+0000","last_clean":"2024-09-10T18:21:52.019286+0000","last_became_active":"2024-09-10T18:21:42.195212+0000","last_became_peered":"2024-09-10T18:21:42.195212+0000","last_unstale":"2024-09-10T18:21:52.019286+0000","last_undegraded":"2024-09-10T18:21:52.019286+0000","last_fullsized":"2024-09-10T18:21:52.019286+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:36:46.215943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.004578+0000","last_change":"2024-09-10T18:21:44.216062+0000","last_active":"2024-09-10T18:21:52.004578+0000","last_peered":"2024-09-10T18:21:52.004578+0000","last_clean":"2024-09-10T18:21:52.004578+0000","last_became_active":"2024-09-10T18:21:44.215710+0000","last_became_peered":"2024-09-10T18:21:44.215710+0000","last_unstale":"2024-09-10T18:21:52.004578+0000","last_undegraded":"2024-09-10T18:21:52.004578+0000","last_fullsized":"2024-09-10T18:21:52.004578+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:39.612550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:53.104222+0000","last_change":"2024-09-10T18:21:46.213003+0000","last_active":"2024-09-10T18:21:53.104222+0000","last_peered":"2024-09-10T18:21:53.104222+0000","last_clean":"2024-09-10T18:21:53.104222+0000","last_became_active":"2024-09-10T18:21:46.212695+0000","last_became_peered":"2024-09-10T18:21:46.212695+0000","last_unstale":"2024-09-10T18:21:53.104222+0000","last_undegraded":"2024-09-10T18:21:53.104222+0000","last_fullsized":"2024-09-10T18:21:53.104222+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:47:09.906108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.005513+0000","last_change":"2024-09-10T18:21:48.248269+0000","last_active":"2024-09-10T18:21:52.005513+0000","last_peered":"2024-09-10T18:21:52.005513+0000","last_clean":"2024-09-10T18:21:52.005513+0000","last_became_active":"2024-09-10T18:21:48.247894+0000","last_became_peered":"2024-09-10T18:21:48.247894+0000","last_unstale":"2024-09-10T18:21:52.005513+0000","last_undegraded":"2024-09-10T18:21:52.005513+0000","last_fullsized":"2024-09-10T18:21:52.005513+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:47.199630+0000","last_clean_scrub_stamp":"2024-09-10T18:21:47.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:32:58.512691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.005162+0000","last_change":"2024-09-10T18:21:42.194518+0000","last_active":"2024-09-10T18:21:52.005162+0000","last_peered":"2024-09-10T18:21:52.005162+0000","last_clean":"2024-09-10T18:21:52.005162+0000","last_became_active":"2024-09-10T18:21:42.194192+0000","last_became_peered":"2024-09-10T18:21:42.194192+0000","last_unstale":"2024-09-10T18:21:52.005162+0000","last_undegraded":"2024-09-10T18:21:52.005162+0000","last_fullsized":"2024-09-10T18:21:52.005162+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:41.160612+0000","last_clean_scrub_stamp":"2024-09-10T18:21:41.160612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:08:08.607365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"59'12","reported_seq":39,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.005394+0000","last_change":"2024-09-10T18:21:44.214158+0000","last_active":"2024-09-10T18:21:52.005394+0000","last_peered":"2024-09-10T18:21:52.005394+0000","last_clean":"2024-09-10T18:21:52.005394+0000","last_became_active":"2024-09-10T18:21:44.213808+0000","last_became_peered":"2024-09-10T18:21:44.213808+0000","last_unstale":"2024-09-10T18:21:52.005394+0000","last_undegraded":"2024-09-10T18:21:52.005394+0000","last_fullsized":"2024-09-10T18:21:52.005394+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:43.186295+0000","last_clean_scrub_stamp":"2024-09-10T18:21:43.186295+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:59:44.036230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:21:52.006612+0000","last_change":"2024-09-10T18:21:46.214567+0000","last_active":"2024-09-10T18:21:52.006612+0000","last_peered":"2024-09-10T18:21:52.006612+0000","last_clean":"2024-09-10T18:21:52.006612+0000","last_became_active":"2024-09-10T18:21:46.214274+0000","last_became_peered":"2024-09-10T18:21:46.214274+0000","last_unstale":"2024-09-10T18:21:52.006612+0000","last_undegraded":"2024-09-10T18:21:52.006612+0000","last_fullsized":"2024-09-10T18:21:52.006612+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:21:45.195997+0000","last_clean_scrub_stamp":"2024-09-10T18:21:45.195997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:35.456006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":71,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":48,"seq":206158430230,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298536,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446616,"statfs":{"total":95995035648,"available":95689334784,"internally_reserved":0,"allocated":1212416,"data_stored":823866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.57999999999999996}]},{"osd":1,"last update":"Tue Sep 10 18:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.48799999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.53200000000000003}]},{"osd":3,"last update":"Tue Sep 10 18:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.503},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.51800000000000002}]},{"osd":4,"last update":"Tue Sep 10 18:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.41299999999999998}]},{"osd":5,"last update":"Tue Sep 10 18:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.59999999999999998}]},{"osd":6,"last update":"Tue Sep 10 18:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.56100000000000005}]}]},{"osd":6,"up_from":42,"seq":180388626458,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298516,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446636,"statfs":{"total":95995035648,"available":95689355264,"internally_reserved":0,"allocated":1191936,"data_stored":822685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.59099999999999997}]},{"osd":1,"last update":"Tue Sep 10 18:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.437},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.42399999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.51900000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.38300000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.53800000000000003}]},{"osd":5,"last update":"Tue Sep 10 18:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.49399999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.47299999999999998}]}]},{"osd":5,"up_from":36,"seq":154618822687,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297936,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447216,"statfs":{"total":95995035648,"available":95689949184,"internally_reserved":0,"allocated":593920,"data_stored":231681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:22:45 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.46700000000000003,"15min":0.47499999999999998},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.58999999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.441},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51300000000000001,"15min":0.51700000000000002},"min":{"1min":0.375,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.59899999999999998,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.51800000000000002}]},{"osd":1,"last update":"Tue Sep 10 18:22:45 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.53800000000000003,"15min":0.54500000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.53000000000000003,"15min":0.53700000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.59599999999999997,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.49299999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:22:45 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.48399999999999999,"15min":0.48599999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.58199999999999996,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.44,"5min":0.50700000000000001,"15min":0.51800000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.58399999999999996,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.503}]},{"osd":3,"last update":"Tue Sep 10 18:22:45 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.51700000000000002,"15min":0.52200000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.69199999999999995,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.51800000000000002,"15min":0.52500000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.61099999999999999,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.54600000000000004}]},{"osd":4,"last update":"Tue Sep 10 18:22:45 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.40100000000000002,"15min":0.40500000000000003},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.55500000000000005,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.435,"5min":0.41999999999999998,"15min":0.41799999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.58199999999999996,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.38200000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:22:05 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.41499999999999998}]},{"osd":7,"last update":"Tue Sep 10 18:22:32 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.45000000000000001}]}]},{"osd":4,"up_from":30,"seq":128849018915,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297956,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447196,"statfs":{"total":95995035648,"available":95689928704,"internally_reserved":0,"allocated":614400,"data_stored":232113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.5,"15min":0.503},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51800000000000002,"15min":0.51900000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.81100000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.28000000000000003}]},{"osd":1,"last update":"Tue Sep 10 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.505,"15min":0.51100000000000001},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.52300000000000002,"15min":0.52500000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.32600000000000001}]},{"osd":2,"last update":"Tue Sep 10 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.50800000000000001,"15min":0.50900000000000001},"min":{"1min":0.373,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.67900000000000005,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52000000000000002,"15min":0.52200000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.33800000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54200000000000004,"15min":0.54000000000000004},"min":{"1min":0.34300000000000003,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.54700000000000004,"15min":0.55300000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.752,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.34899999999999998}]},{"osd":5,"last update":"Tue Sep 10 18:22:43 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.36499999999999999,"15min":0.35699999999999998},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.59499999999999997,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.31},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.372,"15min":0.36599999999999999},"min":{"1min":0.29299999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.623,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.29299999999999998}]},{"osd":6,"last update":"Tue Sep 10 18:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.438}]},{"osd":7,"last update":"Tue Sep 10 18:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.371},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.35999999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182441,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297976,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447176,"statfs":{"total":95995035648,"available":95689908224,"internally_reserved":0,"allocated":634880,"data_stored":232826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.38,"15min":0.38700000000000001},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.38200000000000001,"15min":0.38400000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.47399999999999998}]},{"osd":1,"last update":"Tue Sep 10 18:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.41599999999999998,"15min":0.41999999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.54600000000000004,"5min":0.56999999999999995,"15min":0.56999999999999995},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.38,"5min":0.39100000000000001,"15min":0.39300000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.51600000000000001,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.439}]},{"osd":2,"last update":"Tue Sep 10 18:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.432,"15min":0.44},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.40100000000000002,"15min":0.40699999999999997},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.54300000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.45400000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:22:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.52200000000000002,"15min":0.52400000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.54400000000000004}]},{"osd":5,"last update":"Tue Sep 10 18:22:44 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.53200000000000003,"15min":0.53800000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52400000000000002,"15min":0.52600000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.56499999999999995}]},{"osd":6,"last update":"Tue Sep 10 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.504},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.58399999999999996}]},{"osd":7,"last update":"Tue Sep 10 18:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.63300000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411374,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297944,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447208,"statfs":{"total":95995035648,"available":95689940992,"internally_reserved":0,"allocated":602112,"data_stored":232478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:22:38 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.33600000000000002,"15min":0.32200000000000001},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.52100000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.33200000000000002,"15min":0.317},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.41199999999999998}]},{"osd":1,"last update":"Tue Sep 10 18:22:38 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.33500000000000002,"15min":0.31900000000000001},"min":{"1min":0.249,"5min":0.223,"15min":0.223},"max":{"1min":0.496,"5min":0.496,"15min":0.496},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.35999999999999999,"15min":0.35399999999999998},"min":{"1min":0.252,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.51700000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.46700000000000003}]},{"osd":3,"last update":"Tue Sep 10 18:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.377,"15min":0.38},"min":{"1min":0.23699999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.51100000000000001,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.377,"5min":0.38200000000000001,"15min":0.379},"min":{"1min":0.27000000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.48599999999999999,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.497}]},{"osd":4,"last update":"Tue Sep 10 18:22:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.51900000000000002,"15min":0.51700000000000002},"min":{"1min":0.38500000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.69799999999999995}]},{"osd":5,"last update":"Tue Sep 10 18:22:43 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.53600000000000003,"15min":0.53800000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.878},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56100000000000005,"15min":0.56399999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.85699999999999998}]},{"osd":6,"last update":"Tue Sep 10 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.71799999999999997}]},{"osd":7,"last update":"Tue Sep 10 18:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.70599999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297988,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447164,"statfs":{"total":95995035648,"available":95689895936,"internally_reserved":0,"allocated":647168,"data_stored":233043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.32400000000000001,"15min":0.29699999999999999},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.32800000000000001,"15min":0.30299999999999999},"min":{"1min":0.26900000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.24099999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:22:35 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.35999999999999999,"15min":0.35699999999999998},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.437,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35199999999999998,"15min":0.34599999999999997},"min":{"1min":0.254,"5min":0.222,"15min":0.222},"max":{"1min":0.51200000000000001,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.32200000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:23:03 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.40200000000000002,"15min":0.40600000000000003},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.54000000000000004,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.38400000000000001,"15min":0.39000000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.52200000000000002,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.25700000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:22:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.54600000000000004,"15min":0.55200000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.71599999999999997,"5min":0.747,"15min":0.747},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.496,"5min":0.53600000000000003,"15min":0.54300000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.628,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.35799999999999998}]},{"osd":5,"last update":"Tue Sep 10 18:22:44 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54700000000000004,"15min":0.54900000000000004},"min":{"1min":0.40600000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.66200000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.443},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52800000000000002,"15min":0.52900000000000003},"min":{"1min":0.39300000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.70999999999999996,"5min":0.746,"15min":0.746},"last":0.33400000000000002}]},{"osd":6,"last update":"Tue Sep 10 18:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.39100000000000001}]},{"osd":7,"last update":"Tue Sep 10 18:22:29 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.38100000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738424,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298532,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446620,"statfs":{"total":95995035648,"available":95689338880,"internally_reserved":0,"allocated":1208320,"data_stored":823201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 18:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.33500000000000002,"15min":0.31900000000000001},"min":{"1min":0.27200000000000002,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.313},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.34599999999999997,"15min":0.33200000000000002},"min":{"1min":0.23899999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.54300000000000004,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.24099999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:22:38 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.39200000000000002,"15min":0.39400000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.39200000000000002,"15min":0.38400000000000001},"min":{"1min":0.26100000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.19500000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.45700000000000002,"15min":0.42499999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":4.3689999999999998,"5min":4.3689999999999998,"15min":4.3689999999999998},"last":0.251},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.435,"15min":0.42699999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.219}]},{"osd":4,"last update":"Tue Sep 10 18:22:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.50900000000000001,"15min":0.5},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.371},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51800000000000002,"15min":0.51400000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.35399999999999998}]},{"osd":5,"last update":"Tue Sep 10 18:22:42 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.40799999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.51800000000000002,"15min":0.50800000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.433}]},{"osd":6,"last update":"Tue Sep 10 18:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.36699999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.38800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T18:23:10.451 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T18:23:10.451 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T18:23:10.451 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-10T18:23:10.452 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-10T18:23:10.694 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:23:11.399 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:11 smithi150 ceph-mon[32761]: from='client.24599 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:23:11.399 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:11 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:23:11.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:11 smithi115 ceph-mon[41073]: from='client.24599 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:23:11.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:11 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:23:11.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:11 smithi115 ceph-mon[29603]: from='client.24599 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:23:11.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:11 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:23:12.218 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:12 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a[76671]: ts=2024-09-10T18:23:12.094Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003030502s 2024-09-10T18:23:12.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:12 smithi115 ceph-mon[41073]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:12.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:12 smithi115 ceph-mon[29603]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:12.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:12 smithi150 ceph-mon[32761]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:12.623 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T18:23:12.623 INFO:teuthology.orchestra.run.smithi115.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T18:23:13.305 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:13 smithi150 podman[66009]: 2024-09-10 18:23:13.231674074 +0000 UTC m=+0.119529022 container create 92ac408df0e8fa73388979915d6a351b986fe781a6e6bb9f3595aa055ac9368d (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a, com.redhat.component=ubi8-container, build-date=2022-04-29T11:48:11.055897, io.openshift.expose-services=, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.buildah.version=1.32.0, maintainer=Paul Cuzner , release=239.1651231664, vendor=Red Hat, Inc., io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.tags=base rhel8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, description=Ceph Grafana Container, distribution-scope=public, summary=Grafana Container configured for Ceph mgr/dashboard integration, vcs-type=git, name=ubi8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, architecture=x86_64, version=8.5, io.k8s.display-name=Red Hat Universal Base Image 8) 2024-09-10T18:23:13.424 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-10T18:23:13.425 INFO:tasks.cephadm:Setup complete, yielding 2024-09-10T18:23:13.425 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T18:23:13.435 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi115.front.sepia.ceph.com 2024-09-10T18:23:13.435 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a600e180-6fa0-11ef-bcea-c7b262605968 -- bash -c 'set -ex 2024-09-10T18:23:13.436 DEBUG:teuthology.orchestra.run.smithi115:> for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-10T18:23:13.436 DEBUG:teuthology.orchestra.run.smithi115:> do 2024-09-10T18:23:13.436 DEBUG:teuthology.orchestra.run.smithi115:> echo "rotating key for $f" 2024-09-10T18:23:13.436 DEBUG:teuthology.orchestra.run.smithi115:> K=$(ceph auth get-key $f) 2024-09-10T18:23:13.436 DEBUG:teuthology.orchestra.run.smithi115:> NK="$K" 2024-09-10T18:23:13.436 DEBUG:teuthology.orchestra.run.smithi115:> ceph orch daemon rotate-key $f 2024-09-10T18:23:13.436 DEBUG:teuthology.orchestra.run.smithi115:> while [ "$K" == "$NK" ]; do 2024-09-10T18:23:13.436 DEBUG:teuthology.orchestra.run.smithi115:> sleep 5 2024-09-10T18:23:13.436 DEBUG:teuthology.orchestra.run.smithi115:> NK=$(ceph auth get-key $f) 2024-09-10T18:23:13.436 DEBUG:teuthology.orchestra.run.smithi115:> done 2024-09-10T18:23:13.436 DEBUG:teuthology.orchestra.run.smithi115:> done 2024-09-10T18:23:13.436 DEBUG:teuthology.orchestra.run.smithi115:> ' 2024-09-10T18:23:13.683 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:23:13.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:13 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/2206698416' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T18:23:13.880 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:13 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/2206698416' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T18:23:13.880 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:13 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/2206698416' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T18:23:14.297 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 podman[66009]: 2024-09-10 18:23:14.045598635 +0000 UTC m=+0.933453584 container init 92ac408df0e8fa73388979915d6a351b986fe781a6e6bb9f3595aa055ac9368d (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a, maintainer=Paul Cuzner , summary=Grafana Container configured for Ceph mgr/dashboard integration, io.openshift.expose-services=, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.k8s.display-name=Red Hat Universal Base Image 8, build-date=2022-04-29T11:48:11.055897, architecture=x86_64, vendor=Red Hat, Inc., vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vcs-type=git, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, com.redhat.component=ubi8-container, version=8.5, io.buildah.version=1.32.0, distribution-scope=public, name=ubi8, release=239.1651231664, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.tags=base rhel8, description=Ceph Grafana Container) 2024-09-10T18:23:14.297 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 podman[66009]: 2024-09-10 18:23:14.051808784 +0000 UTC m=+0.939663736 container start 92ac408df0e8fa73388979915d6a351b986fe781a6e6bb9f3595aa055ac9368d (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a, version=8.5, distribution-scope=public, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, name=ubi8, architecture=x86_64, io.buildah.version=1.32.0, release=239.1651231664, maintainer=Paul Cuzner , description=Ceph Grafana Container, summary=Grafana Container configured for Ceph mgr/dashboard integration, build-date=2022-04-29T11:48:11.055897, vcs-type=git, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.tags=base rhel8, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, com.redhat.component=ubi8-container, vendor=Red Hat, Inc., io.openshift.expose-services=, io.k8s.display-name=Red Hat Universal Base Image 8) 2024-09-10T18:23:14.297 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=settings t=2024-09-10T18:23:14.210524174Z level=info msg="Starting Grafana" version=9.4.7 commit=4add91f03d branch=HEAD compiled=2023-03-16T23:56:52Z 2024-09-10T18:23:14.297 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=settings t=2024-09-10T18:23:14.210705749Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-09-10T18:23:14.297 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=settings t=2024-09-10T18:23:14.210725429Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-09-10T18:23:14.297 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=settings t=2024-09-10T18:23:14.210733701Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-09-10T18:23:14.298 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=settings t=2024-09-10T18:23:14.210740082Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-09-10T18:23:14.298 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=settings t=2024-09-10T18:23:14.210745939Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-09-10T18:23:14.298 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=settings t=2024-09-10T18:23:14.210751542Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-09-10T18:23:14.298 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=settings t=2024-09-10T18:23:14.210757292Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-09-10T18:23:14.299 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=settings t=2024-09-10T18:23:14.210763158Z level=info msg="Path Home" path=/usr/share/grafana 2024-09-10T18:23:14.299 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=settings t=2024-09-10T18:23:14.210768892Z level=info msg="Path Data" path=/var/lib/grafana 2024-09-10T18:23:14.299 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=settings t=2024-09-10T18:23:14.210775266Z level=info msg="Path Logs" path=/var/log/grafana 2024-09-10T18:23:14.299 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=settings t=2024-09-10T18:23:14.210781068Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-09-10T18:23:14.299 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=settings t=2024-09-10T18:23:14.210786838Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-09-10T18:23:14.299 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=settings t=2024-09-10T18:23:14.210792929Z level=info msg="App mode production" 2024-09-10T18:23:14.299 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=sqlstore t=2024-09-10T18:23:14.210839539Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-09-10T18:23:14.299 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=sqlstore t=2024-09-10T18:23:14.210855004Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-09-10T18:23:14.299 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.211694419Z level=info msg="Starting DB migrations" 2024-09-10T18:23:14.299 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.211766362Z level=info msg="Executing migration" id="create migration_log table" 2024-09-10T18:23:14.299 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.21228529Z level=info msg="Executing migration" id="create user table" 2024-09-10T18:23:14.299 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.21273344Z level=info msg="Executing migration" id="add unique index user.login" 2024-09-10T18:23:14.300 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.213575637Z level=info msg="Executing migration" id="add unique index user.email" 2024-09-10T18:23:14.300 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.214442924Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-09-10T18:23:14.300 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.215094849Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-09-10T18:23:14.300 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.215937936Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-09-10T18:23:14.300 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.216947396Z level=info msg="Executing migration" id="create user table v2" 2024-09-10T18:23:14.300 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.217463824Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-09-10T18:23:14.300 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.218257115Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-09-10T18:23:14.300 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.219177926Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-09-10T18:23:14.300 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.219980421Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-09-10T18:23:14.300 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.220803692Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-09-10T18:23:14.300 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.2212307Z level=info msg="Executing migration" id="Update user table charset" 2024-09-10T18:23:14.300 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.222060344Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-09-10T18:23:14.300 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.222496823Z level=info msg="Executing migration" id="Add missing user data" 2024-09-10T18:23:14.300 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.222920059Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-09-10T18:23:14.301 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.223806748Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-09-10T18:23:14.301 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.224278403Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-09-10T18:23:14.301 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.224794812Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-09-10T18:23:14.301 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.227441155Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-09-10T18:23:14.301 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.230214819Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-09-10T18:23:14.301 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.233200084Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-09-10T18:23:14.301 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.236028082Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-09-10T18:23:14.301 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.23884479Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-09-10T18:23:14.301 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.241491432Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-09-10T18:23:14.301 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.2443161Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-09-10T18:23:14.301 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.246898968Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-09-10T18:23:14.301 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.249692914Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-09-10T18:23:14.301 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.250147958Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-09-10T18:23:14.302 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.250505456Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-09-10T18:23:14.302 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.251183836Z level=info msg="Executing migration" id="create temp_user v2" 2024-09-10T18:23:14.302 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.251575379Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-09-10T18:23:14.302 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.252004518Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-09-10T18:23:14.302 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.252365987Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-09-10T18:23:14.302 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.25275602Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-09-10T18:23:14.302 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.253138771Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-09-10T18:23:14.302 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.253441987Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-09-10T18:23:14.302 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.253796616Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-09-10T18:23:14.302 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.254125231Z level=info msg="Executing migration" id="create star table" 2024-09-10T18:23:14.302 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.254468178Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-09-10T18:23:14.302 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.254926757Z level=info msg="Executing migration" id="create org table v1" 2024-09-10T18:23:14.302 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.255315289Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-09-10T18:23:14.302 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.25574541Z level=info msg="Executing migration" id="create org_user table v1" 2024-09-10T18:23:14.303 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.256118723Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-09-10T18:23:14.303 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.256541239Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-09-10T18:23:14.303 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.257073499Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-09-10T18:23:14.303 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.257463901Z level=info msg="Executing migration" id="Update org table charset" 2024-09-10T18:23:14.303 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.258332897Z level=info msg="Executing migration" id="Update org_user table charset" 2024-09-10T18:23:14.303 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.258673039Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-09-10T18:23:14.303 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.259753082Z level=info msg="Executing migration" id="create dashboard table" 2024-09-10T18:23:14.303 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.260413974Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-09-10T18:23:14.303 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.261417782Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-09-10T18:23:14.303 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.262040824Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-09-10T18:23:14.303 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.262925421Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-09-10T18:23:14.303 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.263729837Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-09-10T18:23:14.303 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.264606549Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-09-10T18:23:14.304 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.265499336Z level=info msg="Executing migration" id="create dashboard v2" 2024-09-10T18:23:14.304 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.266467919Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-09-10T18:23:14.304 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.267232284Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-09-10T18:23:14.304 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.268242443Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-09-10T18:23:14.304 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.268659712Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-09-10T18:23:14.304 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.269467237Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-09-10T18:23:14.304 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.270349196Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-09-10T18:23:14.304 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.271192508Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-09-10T18:23:14.304 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.273804228Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-09-10T18:23:14.305 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.276601391Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-09-10T18:23:14.305 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.279473303Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-09-10T18:23:14.305 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.286437099Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-09-10T18:23:14.305 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.287069934Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-09-10T18:23:14.305 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.289873675Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-09-10T18:23:14.305 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.290200247Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-09-10T18:23:14.305 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.292549942Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-09-10T18:23:14.305 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.293077094Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-09-10T18:23:14.305 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.295357954Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-09-10T18:23:14.305 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.295910508Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-09-10T18:23:14.305 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.296434318Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-09-10T18:23:14.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:14 smithi115 ceph-mon[41073]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:14.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:14 smithi115 ceph-mon[29603]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:14.547 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-mon[32761]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:14.548 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.299260711Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-09-10T18:23:14.548 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.302007375Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-09-10T18:23:14.548 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.30472931Z level=info msg="Executing migration" id="Update dashboard title length" 2024-09-10T18:23:14.548 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.305033717Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-09-10T18:23:14.548 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.307676841Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-09-10T18:23:14.548 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.308086263Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-09-10T18:23:14.548 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.310511343Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-09-10T18:23:14.548 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.313018453Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-09-10T18:23:14.549 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.31346554Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-09-10T18:23:14.549 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.316294118Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-09-10T18:23:14.549 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.319027966Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-09-10T18:23:14.549 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.321550991Z level=info msg="Executing migration" id="Add check_sum column" 2024-09-10T18:23:14.549 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.322088047Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-09-10T18:23:14.549 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.324751546Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-09-10T18:23:14.549 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.325077796Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-09-10T18:23:14.549 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.327297168Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-09-10T18:23:14.549 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.327765879Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-09-10T18:23:14.549 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.329997885Z level=info msg="Executing migration" id="create data_source table" 2024-09-10T18:23:14.549 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.330460931Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-09-10T18:23:14.550 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.333360416Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-09-10T18:23:14.551 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.333814088Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-09-10T18:23:14.551 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.336539769Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-09-10T18:23:14.551 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.339031249Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-09-10T18:23:14.551 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.341716862Z level=info msg="Executing migration" id="create data_source table v2" 2024-09-10T18:23:14.551 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.344275725Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-09-10T18:23:14.551 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.345016072Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-09-10T18:23:14.551 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.346005517Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-09-10T18:23:14.551 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.346740966Z level=info msg="Executing migration" id="Add column with_credentials" 2024-09-10T18:23:14.551 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.347558223Z level=info msg="Executing migration" id="Add secure json data column" 2024-09-10T18:23:14.551 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.348369719Z level=info msg="Executing migration" id="Update data_source table charset" 2024-09-10T18:23:14.551 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.349337785Z level=info msg="Executing migration" id="Update initial version to 1" 2024-09-10T18:23:14.551 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.350119865Z level=info msg="Executing migration" id="Add read_only data column" 2024-09-10T18:23:14.551 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.350921632Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-09-10T18:23:14.552 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.351850869Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-09-10T18:23:14.552 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.352627265Z level=info msg="Executing migration" id="Add uid column" 2024-09-10T18:23:14.552 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.353467677Z level=info msg="Executing migration" id="Update uid value" 2024-09-10T18:23:14.552 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.354173194Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-09-10T18:23:14.552 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.354948057Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-09-10T18:23:14.552 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.355363653Z level=info msg="Executing migration" id="create api_key table" 2024-09-10T18:23:14.552 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.355772027Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-09-10T18:23:14.552 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.356560667Z level=info msg="Executing migration" id="add index api_key.key" 2024-09-10T18:23:14.552 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.357537342Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-09-10T18:23:14.552 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.35828815Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-09-10T18:23:14.552 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.359071864Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-09-10T18:23:14.552 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.359929538Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-09-10T18:23:14.552 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.360324292Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-09-10T18:23:14.553 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.361364395Z level=info msg="Executing migration" id="create api_key table v2" 2024-09-10T18:23:14.553 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.362140573Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-09-10T18:23:14.553 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.362555478Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-09-10T18:23:14.553 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.362964779Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-09-10T18:23:14.553 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.363359037Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-09-10T18:23:14.553 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.363681459Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-09-10T18:23:14.553 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.364026536Z level=info msg="Executing migration" id="Update api_key table charset" 2024-09-10T18:23:14.554 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.364277129Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-09-10T18:23:14.554 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.364904293Z level=info msg="Executing migration" id="Add service account foreign key" 2024-09-10T18:23:14.554 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.365425578Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-09-10T18:23:14.554 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.365736469Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-09-10T18:23:14.554 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.366262431Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-09-10T18:23:14.554 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.366806404Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-09-10T18:23:14.554 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.367179881Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-09-10T18:23:14.554 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.367511719Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-09-10T18:23:14.554 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.367894548Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-09-10T18:23:14.554 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.368293386Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-09-10T18:23:14.554 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.368719799Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-09-10T18:23:14.554 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.369114521Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-09-10T18:23:14.555 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.369371483Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-09-10T18:23:14.555 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.369655922Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-09-10T18:23:14.555 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.370193171Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-09-10T18:23:14.555 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.370737476Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-09-10T18:23:14.555 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.371033693Z level=info msg="Executing migration" id="create quota table v1" 2024-09-10T18:23:14.555 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.371402906Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-09-10T18:23:14.555 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.371832487Z level=info msg="Executing migration" id="Update quota table charset" 2024-09-10T18:23:14.555 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.372113962Z level=info msg="Executing migration" id="create plugin_setting table" 2024-09-10T18:23:14.555 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.372496137Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-09-10T18:23:14.555 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.372901218Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-09-10T18:23:14.555 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.373420511Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-09-10T18:23:14.555 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.373737595Z level=info msg="Executing migration" id="create session table" 2024-09-10T18:23:14.555 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.374141746Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-09-10T18:23:14.555 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.374401184Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-09-10T18:23:14.556 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.374699763Z level=info msg="Executing migration" id="create playlist table v2" 2024-09-10T18:23:14.556 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.375070881Z level=info msg="Executing migration" id="create playlist item table v2" 2024-09-10T18:23:14.556 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.375438983Z level=info msg="Executing migration" id="Update playlist table charset" 2024-09-10T18:23:14.556 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.375730477Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-09-10T18:23:14.556 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.375982904Z level=info msg="Executing migration" id="drop preferences table v2" 2024-09-10T18:23:14.556 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.376244124Z level=info msg="Executing migration" id="drop preferences table v3" 2024-09-10T18:23:14.556 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.376516292Z level=info msg="Executing migration" id="create preferences table v3" 2024-09-10T18:23:14.556 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.376894173Z level=info msg="Executing migration" id="Update preferences table charset" 2024-09-10T18:23:14.556 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.377146979Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-09-10T18:23:14.556 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.377737809Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-09-10T18:23:14.556 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.378039677Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-09-10T18:23:14.556 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.378593058Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-09-10T18:23:14.556 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.379174484Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-09-10T18:23:14.557 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.381295841Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-09-10T18:23:14.557 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.381726139Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-09-10T18:23:14.557 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.384060804Z level=info msg="Executing migration" id="create alert table v1" 2024-09-10T18:23:14.557 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.386778093Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-09-10T18:23:14.557 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.387229529Z level=info msg="Executing migration" id="add index alert state" 2024-09-10T18:23:14.557 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.390248471Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-09-10T18:23:14.557 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.392955688Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-09-10T18:23:14.557 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.395828561Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-09-10T18:23:14.557 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.396383236Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-09-10T18:23:14.557 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.39916968Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-09-10T18:23:14.557 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.401687138Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-09-10T18:23:14.558 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.404455568Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-09-10T18:23:14.558 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.404926072Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-09-10T18:23:14.558 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.407772142Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-09-10T18:23:14.558 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.408172763Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-09-10T18:23:14.558 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.410589839Z level=info msg="Executing migration" id="Add column is_default" 2024-09-10T18:23:14.558 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.411251575Z level=info msg="Executing migration" id="Add column frequency" 2024-09-10T18:23:14.558 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.41317025Z level=info msg="Executing migration" id="Add column send_reminder" 2024-09-10T18:23:14.558 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.413864214Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-10T18:23:14.558 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.416510259Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-10T18:23:14.558 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.419161849Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-10T18:23:14.558 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.421722025Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-10T18:23:14.558 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.422038236Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-10T18:23:14.558 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.424699856Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-10T18:23:14.558 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.427486745Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-10T18:23:14.558 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.428280143Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-10T18:23:14.559 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.428743033Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-10T18:23:14.559 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.429506462Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-10T18:23:14.559 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.430334436Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-10T18:23:14.559 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.431118196Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-10T18:23:14.559 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.431452776Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-10T18:23:14.559 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.432401066Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-10T18:23:14.559 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.433162878Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-10T18:23:14.559 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.433943947Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-10T18:23:14.559 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.434270274Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-10T18:23:14.559 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.43506683Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-10T18:23:14.559 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.435911499Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-10T18:23:14.559 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.43667272Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-10T18:23:14.559 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.437154011Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-10T18:23:14.559 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.437920697Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-10T18:23:14.559 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.438745487Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-10T18:23:14.560 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.439558322Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-10T18:23:14.560 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.440367042Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-10T18:23:14.560 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.441240432Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-10T18:23:14.560 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.442078357Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-10T18:23:14.560 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.442899433Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-10T18:23:14.560 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.443342828Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-10T18:23:14.560 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.444190627Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-10T18:23:14.560 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.445067184Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-10T18:23:14.560 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.445894698Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-10T18:23:14.560 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.446341991Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-10T18:23:14.560 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.447918973Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-10T18:23:14.560 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.448694904Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-10T18:23:14.560 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.449517228Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-10T18:23:14.560 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.449851668Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-10T18:23:14.561 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.450606341Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-10T18:23:14.561 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.450936018Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-10T18:23:14.561 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.451578271Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-10T18:23:14.561 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.452253874Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-10T18:23:14.561 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.45270745Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-10T18:23:14.561 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.455461404Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-10T18:23:14.561 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.458364065Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-10T18:23:14.562 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.461139118Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-10T18:23:14.562 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.463891864Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-10T18:23:14.562 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.464215449Z level=info msg="Executing migration" id="Move region to single row" 2024-09-10T18:23:14.562 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.466923838Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-10T18:23:14.562 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.467546288Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-10T18:23:14.562 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.469676199Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-10T18:23:14.562 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.472319212Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-10T18:23:14.562 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.472774922Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-10T18:23:14.562 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.475529903Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-10T18:23:14.562 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.478241017Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-10T18:23:14.562 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.481172219Z level=info msg="Executing migration" id="create test_data table" 2024-09-10T18:23:14.562 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.481713448Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-10T18:23:14.562 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.484558637Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-10T18:23:14.562 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.487267186Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-10T18:23:14.563 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.490092766Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-10T18:23:14.563 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.49044812Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-10T18:23:14.563 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.493038401Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-10T18:23:14.563 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.495594018Z level=info msg="Executing migration" id="create team table" 2024-09-10T18:23:14.563 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.498263522Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-10T18:23:14.563 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.498876771Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-10T18:23:14.563 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.501559005Z level=info msg="Executing migration" id="create team member table" 2024-09-10T18:23:14.563 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.504213096Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-10T18:23:14.563 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.506861991Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-10T18:23:14.563 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.507352915Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-10T18:23:14.563 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.510129604Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-10T18:23:14.563 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.51272455Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-10T18:23:14.563 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.5152608Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-10T18:23:14.563 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.516042449Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-10T18:23:14.564 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.518576738Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-10T18:23:14.564 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.521228886Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-10T18:23:14.564 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.523834276Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-10T18:23:14.564 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.52429095Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-10T18:23:14.564 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.526822151Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-10T18:23:14.564 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.527456171Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-10T18:23:14.564 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.528313812Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-10T18:23:14.564 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.529098081Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-10T18:23:14.564 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.529922173Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-10T18:23:14.564 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.530738947Z level=info msg="Executing migration" id="create tag table" 2024-09-10T18:23:14.564 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.53153496Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-10T18:23:14.564 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.532315967Z level=info msg="Executing migration" id="create login attempt table" 2024-09-10T18:23:14.564 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.533122982Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-10T18:23:14.564 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.534016835Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-10T18:23:14.565 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.534667599Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-10T18:23:14.565 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.536672432Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-10T18:23:14.565 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.537414266Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-10T18:23:14.565 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.538207604Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-10T18:23:14.565 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.539091498Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-10T18:23:14.565 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.539961126Z level=info msg="Executing migration" id="create user auth table" 2024-09-10T18:23:14.565 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.543609165Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-10T18:23:14.565 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.544201019Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-10T18:23:14.565 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.545020716Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-10T18:23:14.565 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.545790408Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-10T18:23:14.565 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.546545334Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-10T18:23:14.565 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.5473891Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-10T18:23:14.565 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.54825597Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-10T18:23:14.565 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.549035984Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-10T18:23:14.566 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.549919434Z level=info msg="Executing migration" id="create server_lock table" 2024-09-10T18:23:14.566 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.550783456Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-10T18:23:14.800 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.551615657Z level=info msg="Executing migration" id="create user auth token table" 2024-09-10T18:23:14.800 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.5524507Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-10T18:23:14.800 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.553301254Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-10T18:23:14.800 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.554152409Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-10T18:23:14.800 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.555011425Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-10T18:23:14.800 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.555857987Z level=info msg="Executing migration" id="create cache_data table" 2024-09-10T18:23:14.801 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.556649894Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-10T18:23:14.801 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.557635522Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-10T18:23:14.801 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.558499365Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-10T18:23:14.801 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.559135794Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-10T18:23:14.801 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.561998987Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-10T18:23:14.801 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.562530105Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-10T18:23:14.801 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.564766745Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-10T18:23:14.801 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.567611911Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-10T18:23:14.801 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.57035803Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-10T18:23:14.801 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.570700842Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-10T18:23:14.801 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.571173395Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-10T18:23:14.802 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.571582158Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-10T18:23:14.802 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.572072579Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-10T18:23:14.802 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.572558419Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-10T18:23:14.802 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.573380261Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-10T18:23:14.802 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.574002746Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-10T18:23:14.802 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.574515552Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-10T18:23:14.802 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.574988367Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-10T18:23:14.802 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.575422691Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-10T18:23:14.802 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.575933026Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-10T18:23:14.802 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.576231232Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-10T18:23:14.802 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.57669489Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-10T18:23:14.803 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.577129862Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-10T18:23:14.803 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.577547314Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-10T18:23:14.803 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.577985626Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-10T18:23:14.803 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.578797735Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-10T18:23:14.803 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.579227974Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-10T18:23:14.803 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.579672335Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-10T18:23:14.803 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.583098216Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-10T18:23:14.803 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.586517565Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-10T18:23:14.804 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.586988374Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-10T18:23:14.804 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.587409589Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-10T18:23:14.804 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.588271322Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-10T18:23:14.804 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.588749444Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-10T18:23:14.805 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.589631507Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-10T18:23:14.805 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.5906572Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-10T18:23:14.805 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.591308438Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-10T18:23:14.805 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.592225045Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-10T18:23:14.805 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.593189087Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-10T18:23:14.805 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.594045057Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-10T18:23:14.805 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.596695073Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-10T18:23:14.805 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.599346946Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-10T18:23:14.805 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.602165265Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-10T18:23:14.805 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.604810194Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-10T18:23:14.805 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.607525717Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-10T18:23:14.805 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.610529071Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-10T18:23:14.805 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.613249438Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-10T18:23:14.805 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.615960374Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-10T18:23:14.806 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.618917751Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-10T18:23:14.806 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.621405301Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-10T18:23:14.806 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.624341821Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-10T18:23:14.806 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.626973152Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-10T18:23:14.806 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.627334062Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-10T18:23:14.806 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.630008984Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-10T18:23:14.806 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.632763435Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-10T18:23:14.806 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.635587398Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-10T18:23:14.806 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.638118532Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-10T18:23:14.806 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.64092674Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-10T18:23:14.806 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.643632653Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-10T18:23:14.806 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.646222613Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-10T18:23:14.806 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.648870804Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-10T18:23:14.806 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.651734758Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-10T18:23:14.806 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.654471921Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-10T18:23:14.807 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.656945751Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-10T18:23:14.807 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.659657469Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-10T18:23:14.807 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.662277661Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-10T18:23:14.807 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.665139045Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-10T18:23:14.807 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.667676249Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-10T18:23:14.807 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.670341326Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-10T18:23:14.807 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.672955394Z level=info msg="Executing migration" id="create alert_image table" 2024-09-10T18:23:14.807 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.675591846Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-10T18:23:14.807 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.678476656Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-10T18:23:14.807 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.681052623Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-10T18:23:14.807 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.681454559Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-10T18:23:14.807 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.683659099Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-10T18:23:14.807 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.683738693Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-10T18:23:14.808 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.686230849Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-10T18:23:14.808 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.688910599Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-10T18:23:14.808 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.689662677Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-09-10T18:23:14.808 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.689780755Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-09-10T18:23:14.808 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.689888673Z level=info msg="alerts found to migrate" alerts=0 2024-09-10T18:23:14.808 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.69050513Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-10T18:23:14.808 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.691446978Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-10T18:23:14.808 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.692136166Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-10T18:23:14.808 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.693077198Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-10T18:23:14.808 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.69382398Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-10T18:23:14.808 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.69459748Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-10T18:23:14.808 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.695414346Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-10T18:23:14.809 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.696223336Z level=info msg="Executing migration" id="create data_keys table" 2024-09-10T18:23:14.809 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.697095096Z level=info msg="Executing migration" id="create secrets table" 2024-09-10T18:23:14.809 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.697893059Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-10T18:23:14.809 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.702337614Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-10T18:23:14.809 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.703265349Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-10T18:23:14.809 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.704037634Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-10T18:23:14.809 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.708310638Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-10T18:23:14.809 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.712423557Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-10T18:23:14.809 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.713223574Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-10T18:23:14.809 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.713965583Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-10T18:23:14.809 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.714801288Z level=info msg="Executing migration" id="create permission table" 2024-09-10T18:23:14.809 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.715233634Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-10T18:23:14.809 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.716112063Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-10T18:23:14.809 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.716969502Z level=info msg="Executing migration" id="create role table" 2024-09-10T18:23:14.809 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.717965514Z level=info msg="Executing migration" id="add column display_name" 2024-09-10T18:23:14.810 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.719018568Z level=info msg="Executing migration" id="add column group_name" 2024-09-10T18:23:14.810 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.719964917Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-10T18:23:14.810 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.72086202Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-10T18:23:14.810 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.721641547Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-10T18:23:14.810 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.722516001Z level=info msg="Executing migration" id="create team role table" 2024-09-10T18:23:14.810 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.723345507Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-10T18:23:14.810 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.724303241Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-10T18:23:14.810 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.725229267Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-10T18:23:14.810 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.72600646Z level=info msg="Executing migration" id="create user role table" 2024-09-10T18:23:14.810 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.726979075Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-10T18:23:14.810 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.727654198Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-10T18:23:14.810 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.728512352Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-10T18:23:14.810 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.729404593Z level=info msg="Executing migration" id="create builtin role table" 2024-09-10T18:23:14.810 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.731981053Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-10T18:23:14.811 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.734769042Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-10T18:23:14.811 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.73750407Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-10T18:23:14.811 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.74020304Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-10T18:23:14.811 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.742899462Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-10T18:23:14.811 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.758896216Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-10T18:23:14.811 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.759441026Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-10T18:23:14.811 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.760015472Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-10T18:23:14.811 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.760418947Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-10T18:23:14.811 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.760862267Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-10T18:23:14.811 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.761841932Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-10T18:23:14.811 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.764608808Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-10T18:23:14.811 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.765071129Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-10T18:23:14.811 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.765340819Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-10T18:23:14.811 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.765770758Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-10T18:23:14.812 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.766216224Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-10T18:23:14.812 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.766555157Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-10T18:23:14.812 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.766866527Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-10T18:23:14.812 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.767187244Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-10T18:23:14.812 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.767573149Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-10T18:23:14.812 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.768003575Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-10T18:23:14.812 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.768960532Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-10T18:23:14.812 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.769263568Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-10T18:23:14.813 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.769694135Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-10T18:23:14.813 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.770111497Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-10T18:23:14.813 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.770527354Z level=info msg="Executing migration" id="add correlation config column" 2024-09-10T18:23:14.813 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.771505224Z level=info msg="Executing migration" id="create entity_events table" 2024-09-10T18:23:14.813 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.772013605Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-10T18:23:14.813 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.774672221Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-10T18:23:14.813 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.774753061Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-10T18:23:14.813 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.77503585Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-10T18:23:14.813 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.775111519Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-10T18:23:14.813 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.775358321Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-10T18:23:14.813 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.775752722Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-10T18:23:14.813 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.77826Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-10T18:23:14.813 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.778856385Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-10T18:23:14.813 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.781748528Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-10T18:23:14.814 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.784564111Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-10T18:23:14.814 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.787224062Z level=info msg="Executing migration" id="Drop public config table" 2024-09-10T18:23:14.814 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.78998509Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-10T18:23:14.814 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.792791631Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-10T18:23:14.814 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.795542489Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-10T18:23:14.814 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.798260904Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-10T18:23:14.814 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.800971315Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-10T18:23:15.056 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.804069491Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-10T18:23:15.056 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.806894236Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-10T18:23:15.056 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.809522177Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-10T18:23:15.056 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.81228947Z level=info msg="Executing migration" id="add share column" 2024-09-10T18:23:15.056 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.815176791Z level=info msg="Executing migration" id="create default alerting folders" 2024-09-10T18:23:15.056 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.818084597Z level=info msg="Executing migration" id="create file table" 2024-09-10T18:23:15.057 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.820693482Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-10T18:23:15.057 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.823632595Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-10T18:23:15.057 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.826141239Z level=info msg="Executing migration" id="create file_meta table" 2024-09-10T18:23:15.057 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.828839142Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-10T18:23:15.057 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.831686637Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-10T18:23:15.057 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.834235129Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-10T18:23:15.057 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.837057932Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-10T18:23:15.057 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.839657787Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-10T18:23:15.057 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.842379883Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-10T18:23:15.057 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.8450148Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-10T18:23:15.057 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.847614626Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-10T18:23:15.057 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.850428956Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-10T18:23:15.057 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.853156686Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-10T18:23:15.057 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.855616829Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-10T18:23:15.058 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.858309448Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-10T18:23:15.058 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.860925479Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-10T18:23:15.058 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.863590554Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-10T18:23:15.058 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.865948358Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-10T18:23:15.058 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.875562664Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-10T18:23:15.058 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.876348556Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-10T18:23:15.058 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.877176979Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-10T18:23:15.058 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.880431849Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-10T18:23:15.058 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.881366167Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-10T18:23:15.058 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.882146303Z level=info msg="Executing migration" id="create folder table" 2024-09-10T18:23:15.058 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.882911488Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-10T18:23:15.058 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.883381705Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-10T18:23:15.058 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.883831368Z level=info msg="Executing migration" id="Update folder title length" 2024-09-10T18:23:15.059 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.884630724Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-10T18:23:15.059 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=migrator t=2024-09-10T18:23:14.885541364Z level=info msg="migrations completed" performed=485 skipped=0 duration=673.790595ms 2024-09-10T18:23:15.059 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=sqlstore t=2024-09-10T18:23:14.886020921Z level=info msg="Created default organization" 2024-09-10T18:23:15.059 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 bash[66009]: 92ac408df0e8fa73388979915d6a351b986fe781a6e6bb9f3595aa055ac9368d 2024-09-10T18:23:15.059 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 systemd[1]: Started Ceph grafana.a for a600e180-6fa0-11ef-bcea-c7b262605968. 2024-09-10T18:23:15.059 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=plugin.loader t=2024-09-10T18:23:14.94472073Z level=info msg="Plugin registered" pluginID=input 2024-09-10T18:23:15.059 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=plugin.loader t=2024-09-10T18:23:14.972874863Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-09-10T18:23:15.059 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=plugin.loader t=2024-09-10T18:23:14.972901733Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-09-10T18:23:15.059 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=secrets t=2024-09-10T18:23:14.97294184Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-10T18:23:15.059 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=query_data t=2024-09-10T18:23:14.974627083Z level=info msg="Query Service initialization" 2024-09-10T18:23:15.059 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:14 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=live.push_http t=2024-09-10T18:23:14.976889127Z level=info msg="Live Push Gateway initialization" 2024-09-10T18:23:15.398 INFO:teuthology.orchestra.run.smithi115.stdout:rotating key for osd.0 2024-09-10T18:23:15.398 INFO:teuthology.orchestra.run.smithi115.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-10T18:23:15.398 INFO:teuthology.orchestra.run.smithi115.stderr:+ echo 'rotating key for osd.0' 2024-09-10T18:23:15.398 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:23:15.651 INFO:teuthology.orchestra.run.smithi115.stderr:+ K=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:23:15.651 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:23:15.651 INFO:teuthology.orchestra.run.smithi115.stderr:+ ceph orch daemon rotate-key osd.0 2024-09-10T18:23:15.928 INFO:teuthology.orchestra.run.smithi115.stdout:Scheduled to rotate-key osd.0 on host 'smithi115' 2024-09-10T18:23:15.939 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:23:15.940 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:23:16.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:15 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:16.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:15 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:16.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:15 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:16.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:15 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:16.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:15 smithi150 ceph-mon[32761]: Deploying daemon node-exporter.a on smithi115 2024-09-10T18:23:16.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:15 smithi150 ceph-mon[32761]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:16.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:15 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1597869752' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:23:16.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:15 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:16.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:15 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:16.056 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:15 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=infra.usagestats.collector t=2024-09-10T18:23:15.715958425Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-10T18:23:16.056 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:15 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=provisioning.datasources t=2024-09-10T18:23:15.716275507Z level=warn msg="[Deprecated] the datasource provisioning config is outdated. please upgrade" filename=/etc/grafana/provisioning/datasources/ceph-dashboard.yml 2024-09-10T18:23:16.056 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:15 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=provisioning.datasources t=2024-09-10T18:23:15.748875662Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-10T18:23:16.056 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:15 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=provisioning.datasources t=2024-09-10T18:23:15.780799616Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-09-10T18:23:16.056 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:15 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=provisioning.alerting t=2024-09-10T18:23:15.802057243Z level=info msg="starting to provision alerting" 2024-09-10T18:23:16.057 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:15 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=provisioning.alerting t=2024-09-10T18:23:15.802093689Z level=info msg="finished to provision alerting" 2024-09-10T18:23:16.057 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:15 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=grafanaStorageLogger t=2024-09-10T18:23:15.802231398Z level=info msg="storage starting" 2024-09-10T18:23:16.057 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:15 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=ngalert.state.manager t=2024-09-10T18:23:15.802300031Z level=info msg="Warming state cache for startup" 2024-09-10T18:23:16.057 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:15 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=http.server t=2024-09-10T18:23:15.804134393Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-10T18:23:16.057 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:15 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=sqlstore.transactions t=2024-09-10T18:23:15.812521037Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-10T18:23:16.057 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:15 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=ngalert.state.manager t=2024-09-10T18:23:15.814537612Z level=info msg="State cache has been initialized" states=0 duration=12.236696ms 2024-09-10T18:23:16.057 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:15 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=ticker t=2024-09-10T18:23:15.814593111Z level=info msg=starting first_tick=2024-09-10T18:23:20Z 2024-09-10T18:23:16.057 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:15 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=ngalert.multiorg.alertmanager t=2024-09-10T18:23:15.814599722Z level=info msg="starting MultiOrg Alertmanager" 2024-09-10T18:23:16.057 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:15 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=sqlstore.transactions t=2024-09-10T18:23:15.849217575Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-10T18:23:16.057 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:15 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=sqlstore.transactions t=2024-09-10T18:23:15.889863376Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-10T18:23:16.057 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:23:15 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a[66024]: logger=sqlstore.transactions t=2024-09-10T18:23:15.900419111Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-10T18:23:16.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:15 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:16.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:15 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:16.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:15 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:16.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:15 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:16.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:15 smithi115 ceph-mon[41073]: Deploying daemon node-exporter.a on smithi115 2024-09-10T18:23:16.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:15 smithi115 ceph-mon[41073]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:16.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:15 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1597869752' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:23:16.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:15 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:16.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:15 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:16.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:15 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:16.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:15 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:16.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:15 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:16.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:15 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:16.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:15 smithi115 ceph-mon[29603]: Deploying daemon node-exporter.a on smithi115 2024-09-10T18:23:16.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:15 smithi115 ceph-mon[29603]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:16.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:15 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1597869752' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:23:16.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:15 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:16.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:15 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:16.237 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:16 smithi115 bash[77931]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-10T18:23:16.735 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:16 smithi115 bash[77931]: Getting image source signatures 2024-09-10T18:23:16.735 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:16 smithi115 bash[77931]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-10T18:23:16.735 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:16 smithi115 bash[77931]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-10T18:23:16.735 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:16 smithi115 bash[77931]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-10T18:23:17.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:16 smithi115 ceph-mon[41073]: from='client.24617 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:23:17.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:16 smithi115 ceph-mon[41073]: Schedule rotate-key daemon osd.0 2024-09-10T18:23:17.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:16 smithi115 ceph-mon[29603]: from='client.24617 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:23:17.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:16 smithi115 ceph-mon[29603]: Schedule rotate-key daemon osd.0 2024-09-10T18:23:17.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:16 smithi150 ceph-mon[32761]: from='client.24617 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:23:17.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:16 smithi150 ceph-mon[32761]: Schedule rotate-key daemon osd.0 2024-09-10T18:23:18.187 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:17 smithi115 ceph-mon[29603]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:18.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:17 smithi115 ceph-mon[41073]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:18.187 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:17 smithi115 bash[77931]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-10T18:23:18.187 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:17 smithi115 bash[77931]: Writing manifest to image destination 2024-09-10T18:23:18.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:17 smithi150 ceph-mon[32761]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:18.485 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 podman[77931]: 2024-09-10 18:23:18.160786999 +0000 UTC m=+2.195841066 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-10T18:23:18.485 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 podman[77931]: 2024-09-10 18:23:18.276047986 +0000 UTC m=+2.311102053 container create 5269993361eec018059e5fcdaef113dbeb3f249f4eea9a4fb8d4f543b025a36e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T18:23:18.985 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 podman[77931]: 2024-09-10 18:23:18.577960243 +0000 UTC m=+2.613014316 container init 5269993361eec018059e5fcdaef113dbeb3f249f4eea9a4fb8d4f543b025a36e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T18:23:18.986 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 podman[77931]: 2024-09-10 18:23:18.582815837 +0000 UTC m=+2.617869905 container start 5269993361eec018059e5fcdaef113dbeb3f249f4eea9a4fb8d4f543b025a36e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T18:23:18.986 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.586Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-10T18:23:18.986 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.586Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-10T18:23:18.986 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.587Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-10T18:23:18.986 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.587Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-10T18:23:18.986 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.587Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-10T18:23:18.986 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.587Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-10T18:23:18.986 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-10T18:23:18.986 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=arp 2024-09-10T18:23:18.986 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-10T18:23:18.986 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-10T18:23:18.986 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-10T18:23:18.986 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-10T18:23:18.987 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-10T18:23:18.987 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-10T18:23:18.987 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-10T18:23:18.987 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-10T18:23:18.987 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=edac 2024-09-10T18:23:18.987 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-10T18:23:18.987 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-10T18:23:18.987 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-10T18:23:18.987 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-10T18:23:18.987 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-10T18:23:18.987 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-10T18:23:18.988 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-10T18:23:18.988 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-10T18:23:18.988 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-10T18:23:18.988 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-10T18:23:18.988 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-10T18:23:18.988 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-10T18:23:18.988 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-10T18:23:18.988 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-10T18:23:18.988 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-10T18:23:18.988 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-10T18:23:18.988 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=os 2024-09-10T18:23:18.988 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-10T18:23:18.988 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-10T18:23:18.989 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-10T18:23:18.989 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-10T18:23:18.989 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-10T18:23:18.989 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-10T18:23:18.989 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-10T18:23:18.989 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=stat 2024-09-10T18:23:18.989 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-10T18:23:18.989 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-10T18:23:18.989 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-10T18:23:18.990 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=time 2024-09-10T18:23:18.990 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-10T18:23:18.990 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=uname 2024-09-10T18:23:18.990 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-10T18:23:18.990 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-10T18:23:18.990 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.588Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-10T18:23:18.990 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.589Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-10T18:23:18.990 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a[78007]: ts=2024-09-10T18:23:18.589Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-10T18:23:18.990 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 bash[77931]: 5269993361eec018059e5fcdaef113dbeb3f249f4eea9a4fb8d4f543b025a36e 2024-09-10T18:23:18.990 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:23:18 smithi115 systemd[1]: Started Ceph node-exporter.a for a600e180-6fa0-11ef-bcea-c7b262605968. 2024-09-10T18:23:19.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:23:18 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:23:18] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:23:19.772 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:19 smithi150 bash[66294]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-10T18:23:20.055 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:20 smithi150 bash[66294]: Getting image source signatures 2024-09-10T18:23:20.055 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:20 smithi150 bash[66294]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-10T18:23:20.055 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:20 smithi150 bash[66294]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-10T18:23:20.055 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:20 smithi150 bash[66294]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-10T18:23:20.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:19 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:20.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:19 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:20.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:19 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:20.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:19 smithi150 ceph-mon[32761]: Deploying daemon node-exporter.b on smithi150 2024-09-10T18:23:20.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:19 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2024-09-10T18:23:20.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:19 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2024-09-10T18:23:20.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:19 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2024-09-10T18:23:20.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:19 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-09-10T18:23:20.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:19 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2024-09-10T18:23:20.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:19 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2024-09-10T18:23:20.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:19 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-09-10T18:23:20.057 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:19 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2024-09-10T18:23:20.057 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:19 smithi150 ceph-mon[32761]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:20.057 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:19 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:23:20.057 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:19 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:20.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:19 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:20.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:19 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:20.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:19 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:20.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:19 smithi115 ceph-mon[41073]: Deploying daemon node-exporter.b on smithi150 2024-09-10T18:23:20.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:19 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2024-09-10T18:23:20.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:19 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2024-09-10T18:23:20.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:19 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2024-09-10T18:23:20.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:19 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-09-10T18:23:20.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:19 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2024-09-10T18:23:20.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:19 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2024-09-10T18:23:20.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:19 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-09-10T18:23:20.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:19 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2024-09-10T18:23:20.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:19 smithi115 ceph-mon[41073]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:20.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:19 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:23:20.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:19 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:20.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:19 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:20.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:19 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:20.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:19 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:20.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:19 smithi115 ceph-mon[29603]: Deploying daemon node-exporter.b on smithi150 2024-09-10T18:23:20.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:19 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2024-09-10T18:23:20.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:19 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2024-09-10T18:23:20.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:19 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2024-09-10T18:23:20.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:19 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-09-10T18:23:20.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:19 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2024-09-10T18:23:20.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:19 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2024-09-10T18:23:20.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:19 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-09-10T18:23:20.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:19 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2024-09-10T18:23:20.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:19 smithi115 ceph-mon[29603]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:20.238 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:19 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:23:20.238 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:19 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:20.555 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:23:20 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:23:20.943 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:23:21.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:20 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]': finished 2024-09-10T18:23:21.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:20 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]': finished 2024-09-10T18:23:21.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:20 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]': finished 2024-09-10T18:23:21.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:20 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]': finished 2024-09-10T18:23:21.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:20 smithi150 ceph-mon[32761]: osdmap e62: 8 total, 8 up, 8 in 2024-09-10T18:23:21.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:20 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:23:21.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:20 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]': finished 2024-09-10T18:23:21.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:20 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]': finished 2024-09-10T18:23:21.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:20 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]': finished 2024-09-10T18:23:21.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:20 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]': finished 2024-09-10T18:23:21.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:20 smithi115 ceph-mon[29603]: osdmap e62: 8 total, 8 up, 8 in 2024-09-10T18:23:21.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:20 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:23:21.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:20 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]': finished 2024-09-10T18:23:21.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:20 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]': finished 2024-09-10T18:23:21.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:20 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]': finished 2024-09-10T18:23:21.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:20 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]': finished 2024-09-10T18:23:21.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:20 smithi115 ceph-mon[41073]: osdmap e62: 8 total, 8 up, 8 in 2024-09-10T18:23:21.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:20 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:23:21.255 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:23:21.255 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:23:21.255 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:23:21.792 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:21 smithi150 bash[66294]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-10T18:23:21.793 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:21 smithi150 bash[66294]: Writing manifest to image destination 2024-09-10T18:23:22.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:21 smithi150 ceph-mon[32761]: osdmap e63: 8 total, 8 up, 8 in 2024-09-10T18:23:22.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:21 smithi150 ceph-mon[32761]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:22.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:21 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/4160998289' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:23:22.055 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:21 smithi150 podman[66294]: 2024-09-10 18:23:21.757738188 +0000 UTC m=+2.191715648 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-10T18:23:22.055 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:21 smithi150 podman[66294]: 2024-09-10 18:23:21.839832501 +0000 UTC m=+2.273809990 container create 95530b4a3c723a92014cba595895037f7ac0a800bb7fd3569a7c62769b225c9a (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T18:23:22.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:21 smithi115 ceph-mon[29603]: osdmap e63: 8 total, 8 up, 8 in 2024-09-10T18:23:22.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:21 smithi115 ceph-mon[29603]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:22.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:21 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/4160998289' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:23:22.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:21 smithi115 ceph-mon[41073]: osdmap e63: 8 total, 8 up, 8 in 2024-09-10T18:23:22.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:21 smithi115 ceph-mon[41073]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:22.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:21 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/4160998289' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:23:22.555 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 podman[66294]: 2024-09-10 18:23:22.058789098 +0000 UTC m=+2.492766545 container init 95530b4a3c723a92014cba595895037f7ac0a800bb7fd3569a7c62769b225c9a (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T18:23:22.555 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 podman[66294]: 2024-09-10 18:23:22.063581572 +0000 UTC m=+2.497559029 container start 95530b4a3c723a92014cba595895037f7ac0a800bb7fd3569a7c62769b225c9a (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T18:23:22.555 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.067Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-10T18:23:22.555 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.067Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-10T18:23:22.556 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.067Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-10T18:23:22.556 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.067Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-10T18:23:22.556 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-10T18:23:22.556 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-10T18:23:22.556 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-10T18:23:22.556 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=arp 2024-09-10T18:23:22.556 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-10T18:23:22.556 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-10T18:23:22.556 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-10T18:23:22.556 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-10T18:23:22.556 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-10T18:23:22.556 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-10T18:23:22.557 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-10T18:23:22.557 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-10T18:23:22.557 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=edac 2024-09-10T18:23:22.557 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-10T18:23:22.557 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-10T18:23:22.557 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-10T18:23:22.557 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-10T18:23:22.557 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-10T18:23:22.557 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-10T18:23:22.558 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-10T18:23:22.558 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-10T18:23:22.558 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-10T18:23:22.558 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-10T18:23:22.558 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-10T18:23:22.558 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-10T18:23:22.558 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-10T18:23:22.558 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-10T18:23:22.558 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-10T18:23:22.558 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-10T18:23:22.558 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=os 2024-09-10T18:23:22.558 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-10T18:23:22.558 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-10T18:23:22.558 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-10T18:23:22.558 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-10T18:23:22.559 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-10T18:23:22.559 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-10T18:23:22.559 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-10T18:23:22.559 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=stat 2024-09-10T18:23:22.559 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-10T18:23:22.559 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-10T18:23:22.559 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-10T18:23:22.559 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=time 2024-09-10T18:23:22.559 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-10T18:23:22.559 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=uname 2024-09-10T18:23:22.559 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-10T18:23:22.559 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-10T18:23:22.559 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-10T18:23:22.559 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-10T18:23:22.560 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b[66370]: ts=2024-09-10T18:23:22.068Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-10T18:23:22.560 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 bash[66294]: 95530b4a3c723a92014cba595895037f7ac0a800bb7fd3569a7c62769b225c9a 2024-09-10T18:23:22.560 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:23:22 smithi150 systemd[1]: Started Ceph node-exporter.b for a600e180-6fa0-11ef-bcea-c7b262605968. 2024-09-10T18:23:23.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-mon[32761]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-10T18:23:23.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:23.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:23.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:23.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:23.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:23.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:22 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:23:23.072 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:22 smithi115 ceph-mon[41073]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-10T18:23:23.072 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:22 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:23.072 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:22 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:23.072 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:22 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:23.072 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:22 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:23.072 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:22 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:23.072 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:22 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:23:23.072 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:22 smithi115 ceph-mon[29603]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-10T18:23:23.072 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:22 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:23.072 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:22 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:23.072 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:22 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:23.073 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:22 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:23.073 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:22 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:23.073 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:22 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:23:24.123 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:23 smithi115 ceph-mon[41073]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:23:24.123 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:23 smithi115 ceph-mon[29603]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:23:24.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:23 smithi150 ceph-mon[32761]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:23:25.405 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:25 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:25.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:25 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:25.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:25 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:26.257 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:23:26.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:26 smithi115 ceph-mon[29603]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 13 B/s, 1 objects/s recovering 2024-09-10T18:23:26.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:26 smithi115 ceph-mon[29603]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-10T18:23:26.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:26 smithi115 ceph-mon[29603]: Cluster is now healthy 2024-09-10T18:23:26.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:26 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:26.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:26 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:26.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:26 smithi115 ceph-mon[41073]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 13 B/s, 1 objects/s recovering 2024-09-10T18:23:26.487 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:26 smithi115 ceph-mon[41073]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-10T18:23:26.487 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:26 smithi115 ceph-mon[41073]: Cluster is now healthy 2024-09-10T18:23:26.487 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:26 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:26.487 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:26 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:26.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:26 smithi150 ceph-mon[32761]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 13 B/s, 1 objects/s recovering 2024-09-10T18:23:26.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:26 smithi150 ceph-mon[32761]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-10T18:23:26.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:26 smithi150 ceph-mon[32761]: Cluster is now healthy 2024-09-10T18:23:26.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:26 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:26.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:26 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:26.577 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:23:26.577 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:23:26.577 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:23:27.645 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:27 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:27.646 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:27 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/4263496128' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:23:27.646 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:27 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:27.646 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:27 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:23:27.646 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:27 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:23:27.646 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:27 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:27.646 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:27 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]: dispatch 2024-09-10T18:23:27.646 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:27 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]: dispatch 2024-09-10T18:23:27.646 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:27 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]': finished 2024-09-10T18:23:27.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:27 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:27.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:27 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/4263496128' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:23:27.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:27 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:27.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:27 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:23:27.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:27 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:23:27.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:27 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:27.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:27 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]: dispatch 2024-09-10T18:23:27.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:27 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]: dispatch 2024-09-10T18:23:27.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:27 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]': finished 2024-09-10T18:23:28.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:27 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:28.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:27 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/4263496128' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:23:28.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:27 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:28.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:27 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:23:28.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:27 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:23:28.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:27 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:28.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:27 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]: dispatch 2024-09-10T18:23:28.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:27 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]: dispatch 2024-09-10T18:23:28.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:27 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]': finished 2024-09-10T18:23:28.969 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:28 smithi115 ceph-mon[41073]: Rotating authentication key for osd.0 2024-09-10T18:23:28.969 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:28 smithi115 ceph-mon[41073]: Reconfiguring daemon osd.0 on smithi115 2024-09-10T18:23:28.970 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:28 smithi115 ceph-mon[41073]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 13 B/s, 1 objects/s recovering 2024-09-10T18:23:28.970 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:28 smithi115 ceph-mon[29603]: Rotating authentication key for osd.0 2024-09-10T18:23:28.970 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:28 smithi115 ceph-mon[29603]: Reconfiguring daemon osd.0 on smithi115 2024-09-10T18:23:28.970 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:28 smithi115 ceph-mon[29603]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 13 B/s, 1 objects/s recovering 2024-09-10T18:23:29.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:23:28 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:23:28] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:23:29.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:28 smithi150 ceph-mon[32761]: Rotating authentication key for osd.0 2024-09-10T18:23:29.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:28 smithi150 ceph-mon[32761]: Reconfiguring daemon osd.0 on smithi115 2024-09-10T18:23:29.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:28 smithi150 ceph-mon[32761]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 13 B/s, 1 objects/s recovering 2024-09-10T18:23:29.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:29 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:29.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:29 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:29.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:29 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:29.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:29 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:29.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:29 smithi115 ceph-mon[41073]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-10T18:23:29.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:29 smithi115 ceph-mon[41073]: Reconfiguring daemon alertmanager.a on smithi115 2024-09-10T18:23:29.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:29 smithi115 ceph-mon[41073]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 11 B/s, 1 objects/s recovering 2024-09-10T18:23:29.986 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:29 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:29.986 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:29 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:29.986 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:29 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:29.986 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:29 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:29.986 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:29 smithi115 ceph-mon[29603]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-10T18:23:29.986 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:29 smithi115 ceph-mon[29603]: Reconfiguring daemon alertmanager.a on smithi115 2024-09-10T18:23:29.986 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:29 smithi115 ceph-mon[29603]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 11 B/s, 1 objects/s recovering 2024-09-10T18:23:30.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:29 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:30.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:29 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:30.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:29 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:30.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:29 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:30.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:29 smithi150 ceph-mon[32761]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-10T18:23:30.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:29 smithi150 ceph-mon[32761]: Reconfiguring daemon alertmanager.a on smithi115 2024-09-10T18:23:30.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:29 smithi150 ceph-mon[32761]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 11 B/s, 1 objects/s recovering 2024-09-10T18:23:30.555 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:23:30 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:23:31.228 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:30 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:23:31.228 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:30 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:23:31.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:30 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:23:31.579 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:23:31.900 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:23:31.900 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:23:31.900 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:23:32.116 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:31 smithi115 ceph-mon[41073]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 998 B/s rd, 0 op/s; 10 B/s, 1 objects/s recovering 2024-09-10T18:23:32.116 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:31 smithi115 ceph-mon[29603]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 998 B/s rd, 0 op/s; 10 B/s, 1 objects/s recovering 2024-09-10T18:23:32.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:31 smithi150 ceph-mon[32761]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 998 B/s rd, 0 op/s; 10 B/s, 1 objects/s recovering 2024-09-10T18:23:32.485 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:32 smithi115 systemd[1]: Stopping Ceph alertmanager.a for a600e180-6fa0-11ef-bcea-c7b262605968... 2024-09-10T18:23:32.840 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:32 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a[76671]: ts=2024-09-10T18:23:32.554Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-10T18:23:32.840 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:32 smithi115 podman[79200]: 2024-09-10 18:23:32.574972743 +0000 UTC m=+0.151488799 container died 7146acf5a1ddc98348449435951921525efc965715c4bb600563a024f815b0cb (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:23:33.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:32 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1719923342' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:23:33.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:32 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1719923342' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:23:33.235 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:33 smithi115 podman[79200]: 2024-09-10 18:23:33.035086425 +0000 UTC m=+0.611602480 container cleanup 7146acf5a1ddc98348449435951921525efc965715c4bb600563a024f815b0cb (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:23:33.235 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:33 smithi115 bash[79200]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a 2024-09-10T18:23:33.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:32 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1719923342' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:23:33.595 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:33 smithi115 podman[79217]: 2024-09-10 18:23:33.311342656 +0000 UTC m=+0.738413935 container remove 7146acf5a1ddc98348449435951921525efc965715c4bb600563a024f815b0cb (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:23:33.595 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:33 smithi115 podman[79217]: 2024-09-10 18:23:33.395280311 +0000 UTC m=+0.822351590 volume remove 86b914e57c3a557965815c821201113c99e994e4963bf75968062b420cd924bf 2024-09-10T18:23:33.948 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:33 smithi115 ceph-mon[41073]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 9 B/s, 1 objects/s recovering 2024-09-10T18:23:33.948 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:33 smithi115 ceph-mon[29603]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 9 B/s, 1 objects/s recovering 2024-09-10T18:23:33.948 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:33 smithi115 systemd[1]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-10T18:23:33.948 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:33 smithi115 systemd[1]: Stopped Ceph alertmanager.a for a600e180-6fa0-11ef-bcea-c7b262605968. 2024-09-10T18:23:33.948 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:33 smithi115 systemd[1]: Starting Ceph alertmanager.a for a600e180-6fa0-11ef-bcea-c7b262605968... 2024-09-10T18:23:33.949 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:33 smithi115 podman[79315]: 2024-09-10 18:23:33.715094989 +0000 UTC m=+0.015428994 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-09-10T18:23:33.949 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:33 smithi115 podman[79315]: 2024-09-10 18:23:33.829517623 +0000 UTC m=+0.129851627 volume create c0bde6ca00d4ff184b14ccd00780baa487a8e5351f471fd567f14e2827333b12 2024-09-10T18:23:33.949 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:33 smithi115 podman[79315]: 2024-09-10 18:23:33.946731641 +0000 UTC m=+0.247065684 container create 304ebc95de84e82ae3c5e7ae709d924a70669a072df16b06f1cbd673e863f728 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:23:34.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:33 smithi150 ceph-mon[32761]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 9 B/s, 1 objects/s recovering 2024-09-10T18:23:34.735 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:34 smithi115 podman[79315]: 2024-09-10 18:23:34.2662063 +0000 UTC m=+0.566540300 container init 304ebc95de84e82ae3c5e7ae709d924a70669a072df16b06f1cbd673e863f728 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:23:34.735 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:34 smithi115 podman[79315]: 2024-09-10 18:23:34.27106521 +0000 UTC m=+0.571399216 container start 304ebc95de84e82ae3c5e7ae709d924a70669a072df16b06f1cbd673e863f728 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:23:34.735 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:34 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a[79329]: ts=2024-09-10T18:23:34.282Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-10T18:23:34.735 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:34 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a[79329]: ts=2024-09-10T18:23:34.283Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-10T18:23:34.735 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:34 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a[79329]: ts=2024-09-10T18:23:34.284Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.115 port=9094 2024-09-10T18:23:34.736 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:34 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a[79329]: ts=2024-09-10T18:23:34.285Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-10T18:23:34.736 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:34 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a[79329]: ts=2024-09-10T18:23:34.308Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-10T18:23:34.736 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:34 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a[79329]: ts=2024-09-10T18:23:34.309Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-10T18:23:34.736 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:34 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a[79329]: ts=2024-09-10T18:23:34.311Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-10T18:23:34.736 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:34 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a[79329]: ts=2024-09-10T18:23:34.311Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-10T18:23:34.736 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:34 smithi115 bash[79315]: 304ebc95de84e82ae3c5e7ae709d924a70669a072df16b06f1cbd673e863f728 2024-09-10T18:23:34.736 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:34 smithi115 systemd[1]: Started Ceph alertmanager.a for a600e180-6fa0-11ef-bcea-c7b262605968. 2024-09-10T18:23:35.032 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:34 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:23:35.032 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:34 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:35.032 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:34 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:35.032 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:34 smithi150 ceph-mon[32761]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-10T18:23:35.032 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:34 smithi150 ceph-mon[32761]: Reconfiguring daemon prometheus.a on smithi150 2024-09-10T18:23:35.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:34 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:23:35.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:34 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:35.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:34 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:35.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:34 smithi115 ceph-mon[41073]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-10T18:23:35.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:34 smithi115 ceph-mon[41073]: Reconfiguring daemon prometheus.a on smithi150 2024-09-10T18:23:35.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:34 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:23:35.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:34 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:35.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:34 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:35.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:34 smithi115 ceph-mon[29603]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-10T18:23:35.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:34 smithi115 ceph-mon[29603]: Reconfiguring daemon prometheus.a on smithi150 2024-09-10T18:23:35.989 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:35 smithi150 ceph-mon[32761]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 9 B/s, 1 objects/s recovering 2024-09-10T18:23:36.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:35 smithi115 ceph-mon[41073]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 9 B/s, 1 objects/s recovering 2024-09-10T18:23:36.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:35 smithi115 ceph-mon[29603]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 9 B/s, 1 objects/s recovering 2024-09-10T18:23:36.735 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:36 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a[79329]: ts=2024-09-10T18:23:36.286Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000840696s 2024-09-10T18:23:36.903 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:23:37.222 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:23:37.222 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:23:37.222 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:23:37.654 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:37 smithi150 systemd[1]: Stopping Ceph prometheus.a for a600e180-6fa0-11ef-bcea-c7b262605968... 2024-09-10T18:23:37.961 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:37 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[64035]: ts=2024-09-10T18:23:37.651Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-10T18:23:37.961 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:37 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[64035]: ts=2024-09-10T18:23:37.652Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-10T18:23:37.961 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:37 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[64035]: ts=2024-09-10T18:23:37.652Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-10T18:23:37.961 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:37 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[64035]: ts=2024-09-10T18:23:37.652Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-10T18:23:37.961 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:37 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[64035]: ts=2024-09-10T18:23:37.652Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-10T18:23:37.961 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:37 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[64035]: ts=2024-09-10T18:23:37.652Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-10T18:23:37.961 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:37 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[64035]: ts=2024-09-10T18:23:37.652Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-10T18:23:37.961 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:37 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[64035]: ts=2024-09-10T18:23:37.652Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-10T18:23:37.962 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:37 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[64035]: ts=2024-09-10T18:23:37.652Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-10T18:23:37.962 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:37 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[64035]: ts=2024-09-10T18:23:37.653Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-10T18:23:37.962 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:37 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[64035]: ts=2024-09-10T18:23:37.653Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-10T18:23:37.963 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:37 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[64035]: ts=2024-09-10T18:23:37.653Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-10T18:23:37.963 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:37 smithi150 podman[67413]: 2024-09-10 18:23:37.662323531 +0000 UTC m=+0.113924848 container died f58a458762247727188bf35cb7c96f9b8305576ba3c456ef2181b06971bf1a6c (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:23:38.253 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:38 smithi150 ceph-mon[32761]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:38.253 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:38 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/3618099985' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:23:38.253 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:38 smithi150 podman[67413]: 2024-09-10 18:23:38.075039294 +0000 UTC m=+0.526640613 container cleanup f58a458762247727188bf35cb7c96f9b8305576ba3c456ef2181b06971bf1a6c (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:23:38.253 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:38 smithi150 bash[67413]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a 2024-09-10T18:23:38.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:38 smithi115 ceph-mon[41073]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:38.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:38 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/3618099985' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:23:38.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:38 smithi115 ceph-mon[29603]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:38.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:38 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/3618099985' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:23:38.555 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:38 smithi150 podman[67430]: 2024-09-10 18:23:38.251456019 +0000 UTC m=+0.590726371 container remove f58a458762247727188bf35cb7c96f9b8305576ba3c456ef2181b06971bf1a6c (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:23:38.555 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:38 smithi150 systemd[1]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-09-10T18:23:38.555 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:38 smithi150 systemd[1]: Stopped Ceph prometheus.a for a600e180-6fa0-11ef-bcea-c7b262605968. 2024-09-10T18:23:38.555 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:38 smithi150 systemd[1]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968@prometheus.a.service: Consumed 1.088s CPU time. 2024-09-10T18:23:38.555 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:38 smithi150 systemd[1]: Starting Ceph prometheus.a for a600e180-6fa0-11ef-bcea-c7b262605968... 2024-09-10T18:23:38.964 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:38 smithi150 podman[67529]: 2024-09-10 18:23:38.598691269 +0000 UTC m=+0.016604270 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-10T18:23:38.964 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:38 smithi150 podman[67529]: 2024-09-10 18:23:38.702324145 +0000 UTC m=+0.120237145 container create c79b961f5b3989fd58dd376cec399bc4747a662e805336e650dd7969e8a923a4 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:23:39.306 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:38 smithi150 podman[67529]: 2024-09-10 18:23:38.962803478 +0000 UTC m=+0.380716478 container init c79b961f5b3989fd58dd376cec399bc4747a662e805336e650dd7969e8a923a4 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:23:39.306 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:38 smithi150 podman[67529]: 2024-09-10 18:23:38.967717051 +0000 UTC m=+0.385630045 container start c79b961f5b3989fd58dd376cec399bc4747a662e805336e650dd7969e8a923a4 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:23:39.306 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:38 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[67544]: ts=2024-09-10T18:23:38.991Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-10T18:23:39.306 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:38 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[67544]: ts=2024-09-10T18:23:38.991Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-10T18:23:39.306 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:38 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[67544]: ts=2024-09-10T18:23:38.991Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi150 (none))" 2024-09-10T18:23:39.306 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:38 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[67544]: ts=2024-09-10T18:23:38.991Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-10T18:23:39.306 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:38 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[67544]: ts=2024-09-10T18:23:38.991Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-10T18:23:39.306 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:38 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[67544]: ts=2024-09-10T18:23:38.993Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-10T18:23:39.306 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:38 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[67544]: ts=2024-09-10T18:23:38.993Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-10T18:23:39.306 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:38 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[67544]: ts=2024-09-10T18:23:38.994Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-10T18:23:39.306 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:38 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[67544]: ts=2024-09-10T18:23:38.994Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-10T18:23:39.306 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:38 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[67544]: ts=2024-09-10T18:23:38.996Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-10T18:23:39.307 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:38 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[67544]: ts=2024-09-10T18:23:38.996Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=3.392µs 2024-09-10T18:23:39.307 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:38 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[67544]: ts=2024-09-10T18:23:38.996Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-10T18:23:39.307 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:38 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[67544]: ts=2024-09-10T18:23:38.997Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-09-10T18:23:39.307 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:38 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[67544]: ts=2024-09-10T18:23:38.997Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-09-10T18:23:39.307 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:38 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[67544]: ts=2024-09-10T18:23:38.997Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=41.266µs wal_replay_duration=687.049µs wbl_replay_duration=122ns total_replay_duration=874.059µs 2024-09-10T18:23:39.307 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:38 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[67544]: ts=2024-09-10T18:23:38.998Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-10T18:23:39.307 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:38 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[67544]: ts=2024-09-10T18:23:38.998Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-10T18:23:39.307 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:38 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[67544]: ts=2024-09-10T18:23:38.998Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-10T18:23:39.307 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:39 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[67544]: ts=2024-09-10T18:23:39.014Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=15.811708ms db_storage=1.156µs remote_storage=1.028µs web_handler=343ns query_engine=485ns scrape=375.987µs scrape_sd=25.968µs notify=31.472µs notify_sd=6.583µs rules=15.138751ms tracing=5.547µs 2024-09-10T18:23:39.307 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:39 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[67544]: ts=2024-09-10T18:23:39.014Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-10T18:23:39.307 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:39 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[67544]: ts=2024-09-10T18:23:39.014Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-10T18:23:39.307 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:39 smithi150 bash[67529]: c79b961f5b3989fd58dd376cec399bc4747a662e805336e650dd7969e8a923a4 2024-09-10T18:23:39.307 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:23:39 smithi150 systemd[1]: Started Ceph prometheus.a for a600e180-6fa0-11ef-bcea-c7b262605968. 2024-09-10T18:23:39.428 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:23:39 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: [10/Sep/2024:18:23:39] ENGINE Bus STOPPING 2024-09-10T18:23:39.428 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:23:39 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: [10/Sep/2024:18:23:39] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:23:39.428 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:23:39 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: [10/Sep/2024:18:23:39] ENGINE Bus STOPPED 2024-09-10T18:23:39.735 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:23:39 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: [10/Sep/2024:18:23:39] ENGINE Bus STARTING 2024-09-10T18:23:39.735 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:23:39 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: [10/Sep/2024:18:23:39] ENGINE Serving on http://:::9283 2024-09-10T18:23:39.735 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:23:39 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: [10/Sep/2024:18:23:39] ENGINE Bus STARTED 2024-09-10T18:23:39.735 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:23:39 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: [10/Sep/2024:18:23:39] ENGINE Bus STOPPING 2024-09-10T18:23:40.485 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: [10/Sep/2024:18:23:40] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:23:40.485 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: [10/Sep/2024:18:23:40] ENGINE Bus STOPPED 2024-09-10T18:23:40.485 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: [10/Sep/2024:18:23:40] ENGINE Bus STARTING 2024-09-10T18:23:40.485 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: [10/Sep/2024:18:23:40] ENGINE Serving on http://:::9283 2024-09-10T18:23:40.486 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: [10/Sep/2024:18:23:40] ENGINE Bus STARTED 2024-09-10T18:23:40.486 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: [10/Sep/2024:18:23:40] ENGINE Bus STOPPING 2024-09-10T18:23:40.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[29603]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:40.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:40.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:40.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:23:40.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[29603]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:23:40.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi115.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:23:40.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[29603]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi115.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:23:40.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:40.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:23:40.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[29603]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:23:40.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi150.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:23:40.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[29603]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi150.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:23:40.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:40.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:23:40.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[29603]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:23:40.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi150.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:23:40.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[29603]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi150.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:23:40.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:40.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:23:40.487 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[41073]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:40.487 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:40.488 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:40.488 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:23:40.488 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[41073]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:23:40.488 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi115.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:23:40.488 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[41073]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi115.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:23:40.488 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:40.488 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:23:40.488 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[41073]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:23:40.488 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi150.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:23:40.488 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[41073]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi150.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:23:40.488 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:40.488 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:23:40.488 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[41073]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:23:40.489 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi150.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:23:40.489 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[41073]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi150.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:23:40.489 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:40.489 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:23:40.555 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:23:40 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:23:40.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:40 smithi150 ceph-mon[32761]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:40.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:40 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:40.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:40 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:40.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:40 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:23:40.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:40 smithi150 ceph-mon[32761]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:23:40.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:40 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi115.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:23:40.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:40 smithi150 ceph-mon[32761]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi115.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:23:40.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:40 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:40.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:40 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:23:40.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:40 smithi150 ceph-mon[32761]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:23:40.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:40 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi150.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:23:40.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:40 smithi150 ceph-mon[32761]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi150.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:23:40.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:40 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:40.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:40 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:23:40.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:40 smithi150 ceph-mon[32761]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:23:40.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:40 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi150.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:23:40.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:40 smithi150 ceph-mon[32761]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi150.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:23:40.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:40 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:40.557 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:40 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:23:40.966 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: [10/Sep/2024:18:23:40] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:23:40.966 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: [10/Sep/2024:18:23:40] ENGINE Bus STOPPED 2024-09-10T18:23:40.966 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: [10/Sep/2024:18:23:40] ENGINE Bus STARTING 2024-09-10T18:23:40.966 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: [10/Sep/2024:18:23:40] ENGINE Serving on http://:::9283 2024-09-10T18:23:40.966 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:23:40 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: [10/Sep/2024:18:23:40] ENGINE Bus STARTED 2024-09-10T18:23:41.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:41 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:23:41.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:41 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:23:41.373 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:41 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:23:42.224 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:23:42.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:42 smithi115 ceph-mon[41073]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:42.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:42 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:42.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:42 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:42.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:42 smithi115 ceph-mon[29603]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:42.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:42 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:42.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:42 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:42.529 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:23:42.529 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:23:42.529 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:23:42.554 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:42 smithi150 ceph-mon[32761]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:42.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:42 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:42.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:42 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:43.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:43 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1168208632' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:23:43.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:43 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1168208632' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:23:43.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:43 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1168208632' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:23:44.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:44 smithi115 ceph-mon[41073]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:44.497 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:44 smithi115 ceph-mon[29603]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:44.498 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:23:44 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a[79329]: ts=2024-09-10T18:23:44.288Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003267688s 2024-09-10T18:23:44.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:44 smithi150 ceph-mon[32761]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:45.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:23:44 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:23:44] "GET /metrics HTTP/1.1" 200 219832 "" "Prometheus/2.43.0" 2024-09-10T18:23:46.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:46 smithi115 ceph-mon[29603]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:46.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:46 smithi115 ceph-mon[41073]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:46.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:46 smithi150 ceph-mon[32761]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:47.531 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:23:47.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:47 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:47.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:47 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:47.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:47 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:23:47.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:47 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:23:47.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:47 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:47.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:47 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:23:47.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:47 smithi150 ceph-mon[32761]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:47.874 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:23:47.874 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:23:47.874 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:23:47.874 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:47 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:47.874 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:47 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:47.875 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:47 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:23:47.875 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:47 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:23:47.875 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:47 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:47.875 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:47 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:23:47.875 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:47 smithi115 ceph-mon[41073]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:47.875 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:47 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:47.875 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:47 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:47.875 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:47 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:23:47.875 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:47 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:23:47.875 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:47 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:47.875 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:47 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:23:47.876 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:47 smithi115 ceph-mon[29603]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:48.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:48 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/531977102' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:23:48.688 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:48 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/531977102' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:23:49.021 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:48 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/531977102' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:23:49.021 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:23:48 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:23:48] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:23:49.902 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:49 smithi150 ceph-mon[32761]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:49.902 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:49 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:23:49.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:49 smithi115 ceph-mon[41073]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:49.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:49 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:23:49.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:49 smithi115 ceph-mon[29603]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:49.986 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:49 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:23:50.305 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:23:50 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:23:50.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:50 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:50.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:50 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:50.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:50 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:23:50.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:50 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:50.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:50 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:50.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:50 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:23:50.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:50 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:50.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:50 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:50.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:50 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:23:52.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:51 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:52.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:51 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:52.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:51 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:23:52.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:51 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:23:52.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:51 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:52.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:51 smithi115 ceph-mon[41073]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:52.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:51 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:52.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:51 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:52.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:51 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:23:52.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:51 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:23:52.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:51 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:52.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:51 smithi115 ceph-mon[29603]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:52.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:51 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:52.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:51 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:52.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:51 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:23:52.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:51 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:23:52.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:51 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:23:52.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:51 smithi150 ceph-mon[32761]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:52.876 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:23:53.195 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:23:53.195 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:23:53.195 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:23:54.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:54 smithi115 ceph-mon[29603]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:54.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:54 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/2436187699' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:23:54.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:54 smithi115 ceph-mon[41073]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:54.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:54 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/2436187699' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:23:54.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:54 smithi150 ceph-mon[32761]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:54.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:54 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/2436187699' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:23:55.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:23:54 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:23:54] "GET /metrics HTTP/1.1" 200 219823 "" "Prometheus/2.43.0" 2024-09-10T18:23:56.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:56 smithi115 ceph-mon[41073]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:56.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:56 smithi115 ceph-mon[29603]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:56.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:56 smithi150 ceph-mon[32761]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:58.196 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:23:58.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:58 smithi115 ceph-mon[41073]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:58.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:58 smithi115 ceph-mon[29603]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:58.509 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:23:58.509 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:23:58.509 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:23:58.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:58 smithi150 ceph-mon[32761]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:23:59.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:23:58 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:23:58] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:23:59.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:23:59 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1920384930' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:23:59.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:23:59 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1920384930' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:23:59.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:23:59 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1920384930' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:24:00.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:00 smithi115 ceph-mon[41073]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:00.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:00 smithi115 ceph-mon[29603]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:00.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:00 smithi150 ceph-mon[32761]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:00.555 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:24:00 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:24:01.456 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:01 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:24:01.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:01 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:24:01.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:01 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:24:02.483 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:02 smithi150 ceph-mon[32761]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:02.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:02 smithi115 ceph-mon[41073]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:02.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:02 smithi115 ceph-mon[29603]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:03.512 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:24:03.825 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:24:03.825 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:24:03.825 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:24:04.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:04 smithi115 ceph-mon[41073]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:04.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:04 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1902799991' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:24:04.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:04 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:24:04.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:04 smithi115 ceph-mon[29603]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:04.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:04 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1902799991' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:24:04.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:04 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:24:04.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:04 smithi150 ceph-mon[32761]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:04.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:04 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1902799991' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:24:04.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:04 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:24:05.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:24:04 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:24:04] "GET /metrics HTTP/1.1" 200 219794 "" "Prometheus/2.43.0" 2024-09-10T18:24:06.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:06 smithi115 ceph-mon[41073]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:06.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:06 smithi115 ceph-mon[29603]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:06.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:06 smithi150 ceph-mon[32761]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:08.488 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:08 smithi115 ceph-mon[41073]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:08.489 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:08 smithi115 ceph-mon[29603]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:08.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:08 smithi150 ceph-mon[32761]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:08.827 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:24:09.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:24:08 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:24:08] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:24:09.145 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:24:09.145 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:24:09.145 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:24:09.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:09 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/242197764' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:24:09.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:09 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/242197764' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:24:09.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:09 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/242197764' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:24:10.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:10 smithi115 ceph-mon[29603]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:10.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:10 smithi115 ceph-mon[41073]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:10.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:10 smithi150 ceph-mon[32761]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:10.555 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:24:10 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:24:11.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:11 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:24:11.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:11 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:24:11.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:11 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:24:12.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:12 smithi115 ceph-mon[29603]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:12.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:12 smithi115 ceph-mon[41073]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:12.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:12 smithi150 ceph-mon[32761]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:14.147 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:24:14.463 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:24:14.463 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:24:14.463 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:24:14.463 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:14 smithi115 ceph-mon[41073]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:14.463 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:14 smithi115 ceph-mon[29603]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:14.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:14 smithi150 ceph-mon[32761]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:15.167 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:24:14 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:24:14] "GET /metrics HTTP/1.1" 200 219794 "" "Prometheus/2.43.0" 2024-09-10T18:24:15.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:15 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/3113775599' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:24:15.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:15 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/3113775599' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:24:15.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:15 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/3113775599' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:24:16.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:16 smithi115 ceph-mon[29603]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:16.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:16 smithi115 ceph-mon[41073]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:16.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:16 smithi150 ceph-mon[32761]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:18.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:18 smithi115 ceph-mon[41073]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:18.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:18 smithi115 ceph-mon[29603]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:18.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:18 smithi150 ceph-mon[32761]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:19.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:24:18 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:24:18] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:24:19.464 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:24:19.465 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:19 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:24:19.465 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:19 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:24:19.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:19 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:24:19.780 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:24:19.780 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:24:19.780 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:24:20.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:20 smithi115 ceph-mon[41073]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:20.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:20 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/414808134' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:24:20.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:20 smithi115 ceph-mon[29603]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:20.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:20 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/414808134' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:24:20.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:20 smithi150 ceph-mon[32761]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:20.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:20 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/414808134' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:24:20.555 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:24:20 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:24:21.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:21 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:24:21.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:21 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:24:21.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:21 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:24:22.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:22 smithi115 ceph-mon[41073]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:22.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:22 smithi115 ceph-mon[29603]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:22.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:22 smithi150 ceph-mon[32761]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:24.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:24 smithi115 ceph-mon[41073]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:24.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:24 smithi115 ceph-mon[29603]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:24.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:24 smithi150 ceph-mon[32761]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:24.781 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:24:25.096 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:24:25.096 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:24:25.096 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:24:25.096 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:24:24 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:24:24] "GET /metrics HTTP/1.1" 200 219790 "" "Prometheus/2.43.0" 2024-09-10T18:24:25.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:25 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/3861433815' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:24:25.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:25 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/3861433815' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:24:25.554 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:25 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/3861433815' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:24:26.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:26 smithi115 ceph-mon[41073]: pgmap v68: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:26.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:26 smithi115 ceph-mon[29603]: pgmap v68: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:26.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:26 smithi150 ceph-mon[32761]: pgmap v68: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:28.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:28 smithi150 ceph-mon[32761]: pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:28.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:28 smithi115 ceph-mon[41073]: pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:28.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:28 smithi115 ceph-mon[29603]: pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:29.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:24:28 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:24:28] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:24:30.097 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:24:30.411 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:24:30.411 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:24:30.412 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:24:30.412 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:30 smithi115 ceph-mon[29603]: pgmap v70: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:30.412 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:30 smithi115 ceph-mon[41073]: pgmap v70: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:30.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:30 smithi150 ceph-mon[32761]: pgmap v70: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:30.555 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:24:30 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:24:31.456 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:31 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:24:31.457 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:31 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/3704334214' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:24:31.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:31 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:24:31.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:31 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/3704334214' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:24:31.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:31 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:24:31.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:31 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/3704334214' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:24:32.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:32 smithi115 ceph-mon[41073]: pgmap v71: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:32.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:32 smithi115 ceph-mon[29603]: pgmap v71: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:32.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:32 smithi150 ceph-mon[32761]: pgmap v71: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:34.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:34 smithi115 ceph-mon[41073]: pgmap v72: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:34.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:34 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:24:34.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:34 smithi115 ceph-mon[29603]: pgmap v72: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:34.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:34 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:24:34.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:34 smithi150 ceph-mon[32761]: pgmap v72: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:34.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:34 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:24:35.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:24:34 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:24:34] "GET /metrics HTTP/1.1" 200 219795 "" "Prometheus/2.43.0" 2024-09-10T18:24:35.413 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:24:35.725 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:24:35.725 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:24:35.725 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:24:36.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:36 smithi115 ceph-mon[29603]: pgmap v73: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:36.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:36 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1095854371' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:24:36.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:36 smithi115 ceph-mon[41073]: pgmap v73: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:36.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:36 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1095854371' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:24:36.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:36 smithi150 ceph-mon[32761]: pgmap v73: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:36.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:36 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1095854371' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:24:38.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:38 smithi115 ceph-mon[41073]: pgmap v74: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:38.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:38 smithi115 ceph-mon[29603]: pgmap v74: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:38.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:38 smithi150 ceph-mon[32761]: pgmap v74: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:39.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:24:38 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:24:38] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:24:40.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:40 smithi115 ceph-mon[41073]: pgmap v75: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:40.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:40 smithi115 ceph-mon[29603]: pgmap v75: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:40.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:40 smithi150 ceph-mon[32761]: pgmap v75: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:40.555 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:24:40 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:24:40.727 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:24:41.039 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:24:41.039 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:24:41.039 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:24:41.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:41 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:24:41.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:41 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1394094868' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:24:41.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:41 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:24:41.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:41 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1394094868' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:24:41.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:41 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:24:41.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:41 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1394094868' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:24:42.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:42 smithi115 ceph-mon[41073]: pgmap v76: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:42.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:42 smithi115 ceph-mon[29603]: pgmap v76: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:42.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:42 smithi150 ceph-mon[32761]: pgmap v76: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:44.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:44 smithi115 ceph-mon[41073]: pgmap v77: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:44.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:44 smithi115 ceph-mon[29603]: pgmap v77: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:44.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:44 smithi150 ceph-mon[32761]: pgmap v77: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:45.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:24:44 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:24:44] "GET /metrics HTTP/1.1" 200 219795 "" "Prometheus/2.43.0" 2024-09-10T18:24:46.041 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:24:46.358 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:24:46.359 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:24:46.359 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:24:46.359 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:46 smithi115 ceph-mon[29603]: pgmap v78: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:46.359 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:46 smithi115 ceph-mon[41073]: pgmap v78: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:46.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:46 smithi150 ceph-mon[32761]: pgmap v78: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:47.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:47 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/2773479466' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:24:47.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:47 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/2773479466' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:24:47.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:47 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/2773479466' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:24:48.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:48 smithi115 ceph-mon[29603]: pgmap v79: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:48.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:48 smithi115 ceph-mon[41073]: pgmap v79: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:48.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:48 smithi150 ceph-mon[32761]: pgmap v79: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:49.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:24:48 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:24:48] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:24:49.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:49 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:24:49.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:49 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:24:49.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:49 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:24:50.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:50 smithi115 ceph-mon[29603]: pgmap v80: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:50.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:50 smithi115 ceph-mon[41073]: pgmap v80: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:50.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:50 smithi150 ceph-mon[32761]: pgmap v80: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:50.555 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:24:50 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:24:51.360 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:24:51.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:51 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:24:51.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:51 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:24:51.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:51 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:24:51.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:51 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:24:51.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:51 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:24:51.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:51 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:24:51.669 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:24:51.669 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:24:51.669 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:24:52.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:52 smithi115 ceph-mon[41073]: pgmap v81: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:52.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:52 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:24:52.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:52 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:24:52.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:52 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:24:52.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:52 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1594452432' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:24:52.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:52 smithi115 ceph-mon[29603]: pgmap v81: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:52.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:52 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:24:52.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:52 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:24:52.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:52 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:24:52.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:52 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1594452432' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:24:52.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:52 smithi150 ceph-mon[32761]: pgmap v81: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:52.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:52 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:24:52.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:52 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:24:52.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:52 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:24:52.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:52 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1594452432' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:24:54.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:54 smithi115 ceph-mon[29603]: pgmap v82: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:54.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:54 smithi115 ceph-mon[41073]: pgmap v82: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:54.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:54 smithi150 ceph-mon[32761]: pgmap v82: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:55.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:24:54 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:24:54] "GET /metrics HTTP/1.1" 200 219782 "" "Prometheus/2.43.0" 2024-09-10T18:24:56.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:56 smithi115 ceph-mon[29603]: pgmap v83: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:56.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:56 smithi115 ceph-mon[41073]: pgmap v83: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:56.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:56 smithi150 ceph-mon[32761]: pgmap v83: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:56.671 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:24:56.984 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:24:56.984 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:24:56.984 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:24:57.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:57 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/3403160672' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:24:57.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:57 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/3403160672' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:24:57.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:57 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/3403160672' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:24:58.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:24:58 smithi115 ceph-mon[41073]: pgmap v84: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:58.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:24:58 smithi115 ceph-mon[29603]: pgmap v84: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:58.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:24:58 smithi150 ceph-mon[32761]: pgmap v84: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:24:59.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:24:58 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:24:58] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:25:00.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:00 smithi115 ceph-mon[41073]: pgmap v85: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:00.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:00 smithi115 ceph-mon[29603]: pgmap v85: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:00.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:00 smithi150 ceph-mon[32761]: pgmap v85: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:00.555 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:25:00 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:25:01.456 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:01 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:25:01.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:01 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:25:01.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:01 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:25:01.985 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:25:02.304 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:25:02.304 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:25:02.304 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:25:02.304 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:02 smithi115 ceph-mon[29603]: pgmap v86: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:02.305 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:02 smithi115 ceph-mon[41073]: pgmap v86: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:02.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:02 smithi150 ceph-mon[32761]: pgmap v86: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:03.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:03 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/2691120373' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:25:03.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:03 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/2691120373' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:25:03.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:03 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/2691120373' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:25:04.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:04 smithi115 ceph-mon[41073]: pgmap v87: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:04.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:04 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:25:04.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:04 smithi115 ceph-mon[29603]: pgmap v87: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:04.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:04 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:25:04.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:04 smithi150 ceph-mon[32761]: pgmap v87: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:04.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:04 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:25:05.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:25:04 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:25:04] "GET /metrics HTTP/1.1" 200 219788 "" "Prometheus/2.43.0" 2024-09-10T18:25:06.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:06 smithi115 ceph-mon[41073]: pgmap v88: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:06.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:06 smithi115 ceph-mon[29603]: pgmap v88: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:06.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:06 smithi150 ceph-mon[32761]: pgmap v88: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:07.306 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:25:07.617 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:25:07.618 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:25:07.618 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:25:08.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:08 smithi115 ceph-mon[41073]: pgmap v89: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:08.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:08 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/964913775' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:25:08.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:08 smithi115 ceph-mon[29603]: pgmap v89: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:08.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:08 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/964913775' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:25:08.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:08 smithi150 ceph-mon[32761]: pgmap v89: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:08.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:08 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/964913775' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:25:09.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:25:08 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:25:08] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:25:10.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:10 smithi115 ceph-mon[41073]: pgmap v90: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:10.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:10 smithi115 ceph-mon[29603]: pgmap v90: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:10.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:10 smithi150 ceph-mon[32761]: pgmap v90: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:10.555 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:25:10 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:25:11.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:11 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:25:11.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:11 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:25:11.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:11 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:25:12.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:12 smithi115 ceph-mon[41073]: pgmap v91: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:12.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:12 smithi115 ceph-mon[29603]: pgmap v91: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:12.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:12 smithi150 ceph-mon[32761]: pgmap v91: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:12.619 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:25:12.934 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:25:12.934 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:25:12.934 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:25:13.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:13 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1653346265' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:25:13.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:13 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1653346265' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:25:13.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:13 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1653346265' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:25:14.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:14 smithi115 ceph-mon[29603]: pgmap v92: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:14.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:14 smithi115 ceph-mon[41073]: pgmap v92: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:14.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:14 smithi150 ceph-mon[32761]: pgmap v92: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:15.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:25:14 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:25:14] "GET /metrics HTTP/1.1" 200 219788 "" "Prometheus/2.43.0" 2024-09-10T18:25:16.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:16 smithi150 ceph-mon[32761]: pgmap v93: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:16.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:16 smithi115 ceph-mon[41073]: pgmap v93: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:16.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:16 smithi115 ceph-mon[29603]: pgmap v93: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:17.936 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:25:18.249 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:25:18.249 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:25:18.249 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:25:18.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:18 smithi150 ceph-mon[32761]: pgmap v94: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:18.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:18 smithi115 ceph-mon[41073]: pgmap v94: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:18.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:18 smithi115 ceph-mon[29603]: pgmap v94: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:19.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:25:18 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:25:18] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:25:19.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:19 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/3134215260' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:25:19.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:19 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:25:19.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:19 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/3134215260' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:25:19.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:19 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:25:19.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:19 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/3134215260' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:25:19.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:19 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:25:20.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:20 smithi150 ceph-mon[32761]: pgmap v95: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:20.555 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:25:20 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:25:20.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:20 smithi115 ceph-mon[29603]: pgmap v95: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:20.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:20 smithi115 ceph-mon[41073]: pgmap v95: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:21.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:21 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:25:21.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:21 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:25:21.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:21 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:25:22.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:22 smithi150 ceph-mon[32761]: pgmap v96: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:22.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:22 smithi115 ceph-mon[29603]: pgmap v96: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:22.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:22 smithi115 ceph-mon[41073]: pgmap v96: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:23.251 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:25:23.564 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:25:23.564 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:25:23.565 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:25:24.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:24 smithi150 ceph-mon[32761]: pgmap v97: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:25:24.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:24 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/2054353174' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:25:24.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:24 smithi115 ceph-mon[41073]: pgmap v97: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:25:24.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:24 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/2054353174' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:25:24.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:24 smithi115 ceph-mon[29603]: pgmap v97: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:25:24.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:24 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/2054353174' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:25:25.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:25:24 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:25:24] "GET /metrics HTTP/1.1" 200 219787 "" "Prometheus/2.43.0" 2024-09-10T18:25:26.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:26 smithi150 ceph-mon[32761]: pgmap v98: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:26.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:26 smithi115 ceph-mon[41073]: pgmap v98: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:26.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:26 smithi115 ceph-mon[29603]: pgmap v98: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:28.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:28 smithi150 ceph-mon[32761]: pgmap v99: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:25:28.566 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:25:28.567 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:28 smithi115 ceph-mon[41073]: pgmap v99: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:25:28.567 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:28 smithi115 ceph-mon[29603]: pgmap v99: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:25:28.879 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:25:28.879 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:25:28.879 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:25:29.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:25:28 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:25:28] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:25:29.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:29 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/603008215' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:25:29.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:29 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/603008215' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:25:29.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:29 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/603008215' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:25:30.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:30 smithi150 ceph-mon[32761]: pgmap v100: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:25:30.555 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:25:30 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:25:30.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:30 smithi115 ceph-mon[41073]: pgmap v100: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:25:30.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:30 smithi115 ceph-mon[29603]: pgmap v100: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:25:31.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:31 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:25:31.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:31 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:25:31.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:31 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:25:32.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:32 smithi150 ceph-mon[32761]: pgmap v101: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:32.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:32 smithi115 ceph-mon[41073]: pgmap v101: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:32.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:32 smithi115 ceph-mon[29603]: pgmap v101: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:33.881 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:25:34.197 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:25:34.198 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:25:34.198 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:25:34.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:34 smithi115 ceph-mon[41073]: pgmap v102: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:34.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:34 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:25:34.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:34 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/744039709' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:25:34.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:34 smithi115 ceph-mon[29603]: pgmap v102: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:34.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:34 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:25:34.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:34 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/744039709' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:25:34.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:34 smithi150 ceph-mon[32761]: pgmap v102: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:34.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:34 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:25:34.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:34 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/744039709' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:25:35.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:25:34 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:25:34] "GET /metrics HTTP/1.1" 200 219750 "" "Prometheus/2.43.0" 2024-09-10T18:25:36.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:36 smithi150 ceph-mon[32761]: pgmap v103: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:36.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:36 smithi115 ceph-mon[29603]: pgmap v103: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:36.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:36 smithi115 ceph-mon[41073]: pgmap v103: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:38.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:38 smithi150 ceph-mon[32761]: pgmap v104: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:38.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:38 smithi115 ceph-mon[41073]: pgmap v104: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:38.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:38 smithi115 ceph-mon[29603]: pgmap v104: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:39.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:25:38 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:25:38] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:25:39.199 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:25:39.510 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:25:39.510 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:25:39.511 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:25:40.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:40 smithi150 ceph-mon[32761]: pgmap v105: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:40.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:40 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/819715602' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:25:40.555 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:25:40 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:25:40.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:40 smithi115 ceph-mon[41073]: pgmap v105: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:40.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:40 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/819715602' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:25:40.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:40 smithi115 ceph-mon[29603]: pgmap v105: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:40.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:40 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/819715602' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:25:41.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:41 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:25:41.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:41 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:25:41.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:41 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:25:42.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:42 smithi150 ceph-mon[32761]: pgmap v106: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:42.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:42 smithi115 ceph-mon[41073]: pgmap v106: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:42.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:42 smithi115 ceph-mon[29603]: pgmap v106: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:44.512 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:25:44.554 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:44 smithi150 ceph-mon[32761]: pgmap v107: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:44.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:44 smithi115 ceph-mon[41073]: pgmap v107: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:44.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:44 smithi115 ceph-mon[29603]: pgmap v107: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:44.827 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:25:44.827 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:25:44.828 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:25:45.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:25:44 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:25:44] "GET /metrics HTTP/1.1" 200 219750 "" "Prometheus/2.43.0" 2024-09-10T18:25:45.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:45 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1263900289' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:25:45.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:45 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1263900289' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:25:45.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:45 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1263900289' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:25:46.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:46 smithi150 ceph-mon[32761]: pgmap v108: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:46.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:46 smithi115 ceph-mon[41073]: pgmap v108: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:46.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:46 smithi115 ceph-mon[29603]: pgmap v108: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:48.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:48 smithi150 ceph-mon[32761]: pgmap v109: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:48.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:48 smithi115 ceph-mon[41073]: pgmap v109: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:48.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:48 smithi115 ceph-mon[29603]: pgmap v109: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:49.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:25:48 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:25:48] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:25:49.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:49 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:25:49.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:49 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:25:49.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:49 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:25:49.829 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:25:50.141 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:25:50.141 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:25:50.141 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:25:50.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:50 smithi115 ceph-mon[41073]: pgmap v110: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:50.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:50 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1948690871' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:25:50.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:50 smithi115 ceph-mon[29603]: pgmap v110: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:50.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:50 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1948690871' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:25:50.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:50 smithi150 ceph-mon[32761]: pgmap v110: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:50.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:50 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1948690871' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:25:50.555 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:25:50 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:25:51.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:51 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:25:51.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:51 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:25:51.804 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:51 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:25:52.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:52 smithi115 ceph-mon[41073]: pgmap v111: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:52.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:52 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:25:52.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:52 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:25:52.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:52 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:25:52.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:52 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:25:52.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:52 smithi115 ceph-mon[29603]: pgmap v111: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:52.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:52 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:25:52.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:52 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:25:52.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:52 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:25:52.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:52 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:25:52.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:52 smithi150 ceph-mon[32761]: pgmap v111: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:52.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:52 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:25:52.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:52 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:25:52.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:52 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:25:52.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:52 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:25:54.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:54 smithi115 ceph-mon[41073]: pgmap v112: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:54.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:54 smithi115 ceph-mon[29603]: pgmap v112: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:54.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:54 smithi150 ceph-mon[32761]: pgmap v112: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:25:55.143 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:25:55.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:25:54 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:25:54] "GET /metrics HTTP/1.1" 200 219741 "" "Prometheus/2.43.0" 2024-09-10T18:25:55.462 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:25:55.462 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:25:55.462 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:25:56.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:56 smithi115 ceph-mon[41073]: pgmap v113: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:56.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:56 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/3924086363' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:25:56.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:56 smithi115 ceph-mon[29603]: pgmap v113: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:56.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:56 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/3924086363' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:25:56.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:56 smithi150 ceph-mon[32761]: pgmap v113: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:56.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:56 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/3924086363' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:25:58.655 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:25:58 smithi150 ceph-mon[32761]: pgmap v114: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:25:58.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:25:58 smithi115 ceph-mon[41073]: pgmap v114: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:25:58.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:25:58 smithi115 ceph-mon[29603]: pgmap v114: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:25:59.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:25:58 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:25:58] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:26:00.463 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:26:00.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:00 smithi150 ceph-mon[32761]: pgmap v115: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:26:00.555 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:26:00 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:26:00.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:00 smithi115 ceph-mon[41073]: pgmap v115: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:26:00.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:00 smithi115 ceph-mon[29603]: pgmap v115: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:26:00.776 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:26:00.776 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:26:00.776 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:26:01.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:01 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:26:01.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:01 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/2173140205' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:01.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:01 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:26:01.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:01 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/2173140205' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:01.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:01 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:26:01.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:01 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/2173140205' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:02.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:02 smithi115 ceph-mon[41073]: pgmap v116: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:26:02.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:02 smithi115 ceph-mon[29603]: pgmap v116: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:26:02.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:02 smithi150 ceph-mon[32761]: pgmap v116: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:26:04.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:04 smithi115 ceph-mon[41073]: pgmap v117: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:04.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:04 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:26:04.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:04 smithi115 ceph-mon[29603]: pgmap v117: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:04.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:04 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:26:04.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:04 smithi150 ceph-mon[32761]: pgmap v117: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:04.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:04 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:26:05.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:26:04 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:26:04] "GET /metrics HTTP/1.1" 200 219749 "" "Prometheus/2.43.0" 2024-09-10T18:26:05.777 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:26:06.089 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:26:06.089 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:26:06.089 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:26:06.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:06 smithi115 ceph-mon[29603]: pgmap v118: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:26:06.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:06 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/2167340740' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:06.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:06 smithi115 ceph-mon[41073]: pgmap v118: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:26:06.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:06 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/2167340740' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:06.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:06 smithi150 ceph-mon[32761]: pgmap v118: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:26:06.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:06 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/2167340740' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:08.656 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:08 smithi150 ceph-mon[32761]: pgmap v119: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:08.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:08 smithi115 ceph-mon[29603]: pgmap v119: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:08.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:08 smithi115 ceph-mon[41073]: pgmap v119: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:09.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:26:08 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:26:08] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:26:10.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:10 smithi150 ceph-mon[32761]: pgmap v120: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:10.555 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:26:10 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:26:10.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:10 smithi115 ceph-mon[41073]: pgmap v120: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:10.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:10 smithi115 ceph-mon[29603]: pgmap v120: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:11.090 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:26:11.405 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:26:11.405 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:26:11.405 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:26:11.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:11 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:26:11.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:11 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:26:11.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:11 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:26:12.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:12 smithi115 ceph-mon[41073]: pgmap v121: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:26:12.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:12 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/2265846523' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:12.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:12 smithi115 ceph-mon[29603]: pgmap v121: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:26:12.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:12 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/2265846523' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:12.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:12 smithi150 ceph-mon[32761]: pgmap v121: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:26:12.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:12 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/2265846523' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:14.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:14 smithi115 ceph-mon[41073]: pgmap v122: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:14.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:14 smithi115 ceph-mon[29603]: pgmap v122: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:14.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:14 smithi150 ceph-mon[32761]: pgmap v122: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:15.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:26:14 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:26:14] "GET /metrics HTTP/1.1" 200 219749 "" "Prometheus/2.43.0" 2024-09-10T18:26:16.406 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:26:16.722 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:26:16.722 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:26:16.722 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:26:16.722 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:16 smithi115 ceph-mon[41073]: pgmap v123: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:26:16.722 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:16 smithi115 ceph-mon[29603]: pgmap v123: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:26:16.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:16 smithi150 ceph-mon[32761]: pgmap v123: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:26:17.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:17 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/729320725' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:17.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:17 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/729320725' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:17.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:17 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/729320725' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:18.636 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:18 smithi115 ceph-mon[41073]: pgmap v124: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:18.637 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:18 smithi115 ceph-mon[29603]: pgmap v124: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:18.655 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:18 smithi150 ceph-mon[32761]: pgmap v124: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:19.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:26:18 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:26:18] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:26:19.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:19 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:26:19.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:19 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:26:19.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:19 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:26:20.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:20 smithi150 ceph-mon[32761]: pgmap v125: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:20.555 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:26:20 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:26:20.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:20 smithi115 ceph-mon[41073]: pgmap v125: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:20.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:20 smithi115 ceph-mon[29603]: pgmap v125: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:21.724 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:26:21.724 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:21 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:26:21.724 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:21 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:26:21.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:21 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:26:22.040 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:26:22.040 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:26:22.040 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:26:22.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:22 smithi115 ceph-mon[41073]: pgmap v126: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:26:22.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:22 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1040797669' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:22.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:22 smithi115 ceph-mon[29603]: pgmap v126: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:26:22.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:22 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1040797669' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:22.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:22 smithi150 ceph-mon[32761]: pgmap v126: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:26:22.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:22 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1040797669' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:24.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:24 smithi115 ceph-mon[41073]: pgmap v127: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:24.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:24 smithi115 ceph-mon[29603]: pgmap v127: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:24.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:24 smithi150 ceph-mon[32761]: pgmap v127: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:25.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:26:24 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:26:24] "GET /metrics HTTP/1.1" 200 219753 "" "Prometheus/2.43.0" 2024-09-10T18:26:26.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:26 smithi115 ceph-mon[41073]: pgmap v128: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:26:26.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:26 smithi115 ceph-mon[29603]: pgmap v128: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:26:26.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:26 smithi150 ceph-mon[32761]: pgmap v128: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:26:27.042 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:26:27.354 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:26:27.354 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:26:27.354 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:26:27.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:27 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1002317751' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:27.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:27 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1002317751' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:27.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:27 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1002317751' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:28.655 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:28 smithi150 ceph-mon[32761]: pgmap v129: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:28.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:28 smithi115 ceph-mon[41073]: pgmap v129: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:28.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:28 smithi115 ceph-mon[29603]: pgmap v129: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:29.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:26:28 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:26:28] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:26:30.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:30 smithi150 ceph-mon[32761]: pgmap v130: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:30.555 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:26:30 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:26:30.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:30 smithi115 ceph-mon[41073]: pgmap v130: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:30.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:30 smithi115 ceph-mon[29603]: pgmap v130: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:31.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:31 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:26:31.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:31 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:26:31.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:31 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:26:32.356 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:26:32.671 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:26:32.672 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:26:32.672 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:26:32.672 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:32 smithi115 ceph-mon[41073]: pgmap v131: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:26:32.672 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:32 smithi115 ceph-mon[29603]: pgmap v131: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:26:32.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:32 smithi150 ceph-mon[32761]: pgmap v131: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:26:33.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:33 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/2355950468' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:33.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:33 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/2355950468' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:33.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:33 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/2355950468' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:34.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:34 smithi115 ceph-mon[41073]: pgmap v132: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:34.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:34 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:26:34.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:34 smithi115 ceph-mon[29603]: pgmap v132: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:34.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:34 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:26:34.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:34 smithi150 ceph-mon[32761]: pgmap v132: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:34.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:34 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:26:35.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:26:34 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:26:34] "GET /metrics HTTP/1.1" 200 219751 "" "Prometheus/2.43.0" 2024-09-10T18:26:36.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:36 smithi115 ceph-mon[41073]: pgmap v133: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:26:36.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:36 smithi115 ceph-mon[29603]: pgmap v133: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:26:36.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:36 smithi150 ceph-mon[32761]: pgmap v133: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:26:37.673 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:26:37.982 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:26:37.983 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:26:37.983 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:26:38.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:38 smithi115 ceph-mon[41073]: pgmap v134: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:38.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:38 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/3359209311' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:38.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:38 smithi115 ceph-mon[29603]: pgmap v134: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:38.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:38 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/3359209311' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:38.805 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:26:38 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:26:38] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:26:38.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:38 smithi150 ceph-mon[32761]: pgmap v134: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:38.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:38 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/3359209311' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:40.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:40 smithi150 ceph-mon[32761]: pgmap v135: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:40.555 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:26:40 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:26:40.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:40 smithi115 ceph-mon[41073]: pgmap v135: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:40.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:40 smithi115 ceph-mon[29603]: pgmap v135: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:41.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:41 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:26:41.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:41 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:26:41.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:41 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:26:42.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:42 smithi115 ceph-mon[41073]: pgmap v136: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:26:42.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:42 smithi115 ceph-mon[29603]: pgmap v136: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:26:42.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:42 smithi150 ceph-mon[32761]: pgmap v136: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:26:42.985 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:26:43.296 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:26:43.296 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:26:43.296 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:26:43.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:43 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1114743632' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:43.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:43 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1114743632' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:43.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:43 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1114743632' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:44.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:44 smithi115 ceph-mon[41073]: pgmap v137: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:44.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:44 smithi115 ceph-mon[29603]: pgmap v137: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:44.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:44 smithi150 ceph-mon[32761]: pgmap v137: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:45.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:26:44 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:26:44] "GET /metrics HTTP/1.1" 200 219751 "" "Prometheus/2.43.0" 2024-09-10T18:26:46.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:46 smithi115 ceph-mon[29603]: pgmap v138: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:26:46.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:46 smithi115 ceph-mon[41073]: pgmap v138: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:26:46.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:46 smithi150 ceph-mon[32761]: pgmap v138: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:26:48.298 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:26:48.609 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:26:48.609 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:26:48.609 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:26:48.610 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:48 smithi115 ceph-mon[41073]: pgmap v139: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:48.610 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:48 smithi115 ceph-mon[29603]: pgmap v139: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:48.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:48 smithi150 ceph-mon[32761]: pgmap v139: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:48.805 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:26:48 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:26:48] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:26:49.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:49 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1981518426' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:49.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:49 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:26:49.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:49 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1981518426' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:49.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:49 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:26:49.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:49 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1981518426' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:49.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:49 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:26:50.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:50 smithi150 ceph-mon[32761]: pgmap v140: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:50.555 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:26:50 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:26:50.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:50 smithi115 ceph-mon[41073]: pgmap v140: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:50.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:50 smithi115 ceph-mon[29603]: pgmap v140: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:51.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:51 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:26:51.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:51 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:26:51.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:51 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:26:52.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:52 smithi115 ceph-mon[29603]: pgmap v141: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:26:52.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:52 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:26:52.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:52 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:26:52.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:52 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:26:52.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:52 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:26:52.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:52 smithi115 ceph-mon[41073]: pgmap v141: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:26:52.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:52 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:26:52.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:52 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:26:52.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:52 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:26:52.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:52 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:26:52.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:52 smithi150 ceph-mon[32761]: pgmap v141: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:26:52.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:52 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:26:52.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:52 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:26:52.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:52 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:26:52.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:52 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:26:53.611 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:26:53.924 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:26:53.924 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:26:53.924 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:26:54.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:54 smithi115 ceph-mon[41073]: pgmap v142: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:54.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:54 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/2257986762' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:54.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:54 smithi115 ceph-mon[29603]: pgmap v142: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:54.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:54 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/2257986762' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:54.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:54 smithi150 ceph-mon[32761]: pgmap v142: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:54.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:54 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/2257986762' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:55.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:26:54 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:26:54] "GET /metrics HTTP/1.1" 200 219761 "" "Prometheus/2.43.0" 2024-09-10T18:26:56.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:56 smithi115 ceph-mon[41073]: pgmap v143: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:26:56.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:56 smithi115 ceph-mon[29603]: pgmap v143: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:26:56.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:56 smithi150 ceph-mon[32761]: pgmap v143: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:26:58.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:58 smithi115 ceph-mon[29603]: pgmap v144: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:58.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:58 smithi115 ceph-mon[41073]: pgmap v144: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:58.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:58 smithi150 ceph-mon[32761]: pgmap v144: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:26:58.805 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:26:58 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:26:58] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:26:58.925 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:26:59.239 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== 2024-09-10T18:26:59.239 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== ']' 2024-09-10T18:26:59.239 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:26:59.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:26:59 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1407359007' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:59.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:26:59 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1407359007' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:59.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:26:59 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1407359007' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:27:00.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:00 smithi150 ceph-mon[32761]: pgmap v145: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:00.555 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:27:00 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:27:00.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:00 smithi115 ceph-mon[41073]: pgmap v145: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:00.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:00 smithi115 ceph-mon[29603]: pgmap v145: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:01.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:01 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:27:01.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:01 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:27:01.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:01 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:27:02.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:02 smithi115 ceph-mon[29603]: pgmap v146: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:27:02.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:02 smithi115 ceph-mon[41073]: pgmap v146: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:27:02.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:02 smithi150 ceph-mon[32761]: pgmap v146: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:27:04.241 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T18:27:04.549 INFO:teuthology.orchestra.run.smithi115.stdout:rotating key for osd.1 2024-09-10T18:27:04.549 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQCejuBmx9dFOxAAykNGqBYfz/UkwYaaFE1L+Q== 2024-09-10T18:27:04.549 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBxjeBm9y9QORAATihzOVejMmNkZrH1mDJ5ig== == AQCejuBmx9dFOxAAykNGqBYfz/UkwYaaFE1L+Q== ']' 2024-09-10T18:27:04.550 INFO:teuthology.orchestra.run.smithi115.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-10T18:27:04.550 INFO:teuthology.orchestra.run.smithi115.stderr:+ echo 'rotating key for osd.1' 2024-09-10T18:27:04.550 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:04 smithi115 ceph-mon[41073]: pgmap v147: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:04.550 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:04 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:27:04.550 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:04 smithi115 ceph-mon[29603]: pgmap v147: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:04.551 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:04 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:27:04.551 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.1 2024-09-10T18:27:04.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:04 smithi150 ceph-mon[32761]: pgmap v147: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:04.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:04 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:27:04.851 INFO:teuthology.orchestra.run.smithi115.stderr:+ K=AQCJjeBmUId4KxAAN986T3CaaFGHsvLfPCExdA== 2024-09-10T18:27:04.851 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQCJjeBmUId4KxAAN986T3CaaFGHsvLfPCExdA== 2024-09-10T18:27:04.851 INFO:teuthology.orchestra.run.smithi115.stderr:+ ceph orch daemon rotate-key osd.1 2024-09-10T18:27:05.111 INFO:teuthology.orchestra.run.smithi115.stdout:Scheduled to rotate-key osd.1 on host 'smithi115' 2024-09-10T18:27:05.111 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:27:04 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:27:04] "GET /metrics HTTP/1.1" 200 219766 "" "Prometheus/2.43.0" 2024-09-10T18:27:05.122 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQCJjeBmUId4KxAAN986T3CaaFGHsvLfPCExdA== == AQCJjeBmUId4KxAAN986T3CaaFGHsvLfPCExdA== ']' 2024-09-10T18:27:05.122 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:27:05.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:05 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1070592240' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:27:05.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:05 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/3267010545' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T18:27:05.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:05 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:05.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:05 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:05.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:05 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:27:05.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:05 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:27:05.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:05 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:27:05.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:05 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:05.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:05 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]: dispatch 2024-09-10T18:27:05.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:05 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]: dispatch 2024-09-10T18:27:05.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:05 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]': finished 2024-09-10T18:27:05.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:05 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1070592240' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:27:05.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:05 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/3267010545' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T18:27:05.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:05 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:05.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:05 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:05.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:05 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:27:05.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:05 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:27:05.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:05 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:27:05.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:05 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:05.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:05 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]: dispatch 2024-09-10T18:27:05.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:05 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]: dispatch 2024-09-10T18:27:05.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:05 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]': finished 2024-09-10T18:27:05.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:05 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1070592240' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T18:27:05.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:05 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/3267010545' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T18:27:05.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:05 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:05.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:05 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:05.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:05 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:27:05.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:05 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:27:05.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:05 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:27:05.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:05 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:05.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:05 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]: dispatch 2024-09-10T18:27:05.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:05 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]: dispatch 2024-09-10T18:27:05.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:05 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]': finished 2024-09-10T18:27:06.723 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:06 smithi115 ceph-mon[29603]: from='client.24899 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:27:06.723 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:06 smithi115 ceph-mon[29603]: Schedule rotate-key daemon osd.1 2024-09-10T18:27:06.723 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:06 smithi115 ceph-mon[29603]: pgmap v148: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:27:06.723 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:06 smithi115 ceph-mon[29603]: Rotating authentication key for osd.1 2024-09-10T18:27:06.723 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:06 smithi115 ceph-mon[29603]: Reconfiguring daemon osd.1 on smithi115 2024-09-10T18:27:06.723 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:06 smithi115 ceph-mon[41073]: from='client.24899 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:27:06.723 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:06 smithi115 ceph-mon[41073]: Schedule rotate-key daemon osd.1 2024-09-10T18:27:06.724 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:06 smithi115 ceph-mon[41073]: pgmap v148: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:27:06.724 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:06 smithi115 ceph-mon[41073]: Rotating authentication key for osd.1 2024-09-10T18:27:06.724 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:06 smithi115 ceph-mon[41073]: Reconfiguring daemon osd.1 on smithi115 2024-09-10T18:27:06.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:06 smithi150 ceph-mon[32761]: from='client.24899 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:27:06.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:06 smithi150 ceph-mon[32761]: Schedule rotate-key daemon osd.1 2024-09-10T18:27:06.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:06 smithi150 ceph-mon[32761]: pgmap v148: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:27:06.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:06 smithi150 ceph-mon[32761]: Rotating authentication key for osd.1 2024-09-10T18:27:06.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:06 smithi150 ceph-mon[32761]: Reconfiguring daemon osd.1 on smithi115 2024-09-10T18:27:08.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:07 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:08.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:07 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:08.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:07 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:08.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:07 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:08.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:07 smithi115 ceph-mon[41073]: pgmap v149: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:08.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:07 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:08.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:07 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:08.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:07 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:08.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:07 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:08.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:07 smithi115 ceph-mon[29603]: pgmap v149: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:08.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:07 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:08.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:07 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:08.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:07 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:08.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:07 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:08.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:07 smithi150 ceph-mon[32761]: pgmap v149: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:09.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:27:08 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:27:08] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:27:10.125 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.1 2024-09-10T18:27:10.439 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQCJjeBmUId4KxAAN986T3CaaFGHsvLfPCExdA== 2024-09-10T18:27:10.440 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQCJjeBmUId4KxAAN986T3CaaFGHsvLfPCExdA== == AQCJjeBmUId4KxAAN986T3CaaFGHsvLfPCExdA== ']' 2024-09-10T18:27:10.440 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:27:10.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:10 smithi115 ceph-mon[41073]: pgmap v150: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:10.440 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:10 smithi115 ceph-mon[29603]: pgmap v150: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:10.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:10 smithi150 ceph-mon[32761]: pgmap v150: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:10.555 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:27:10 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:27:11.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:11 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:27:11.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:11 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/305307571' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T18:27:11.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:11 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:27:11.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:11 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/305307571' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T18:27:11.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:11 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:27:11.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:11 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/305307571' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T18:27:12.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:12 smithi115 ceph-mon[41073]: pgmap v151: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:27:12.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:12 smithi115 ceph-mon[29603]: pgmap v151: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:27:12.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:12 smithi150 ceph-mon[32761]: pgmap v151: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:27:14.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:14 smithi115 ceph-mon[41073]: pgmap v152: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:14.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:14 smithi115 ceph-mon[29603]: pgmap v152: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:14.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:14 smithi150 ceph-mon[32761]: pgmap v152: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:15.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:27:14 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:27:14] "GET /metrics HTTP/1.1" 200 219766 "" "Prometheus/2.43.0" 2024-09-10T18:27:15.441 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.1 2024-09-10T18:27:15.753 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQCJjeBmUId4KxAAN986T3CaaFGHsvLfPCExdA== 2024-09-10T18:27:15.753 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQCJjeBmUId4KxAAN986T3CaaFGHsvLfPCExdA== == AQCJjeBmUId4KxAAN986T3CaaFGHsvLfPCExdA== ']' 2024-09-10T18:27:15.753 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:27:16.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:16 smithi115 ceph-mon[29603]: pgmap v153: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:27:16.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:16 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/2611213952' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T18:27:16.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:16 smithi115 ceph-mon[41073]: pgmap v153: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:27:16.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:16 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/2611213952' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T18:27:16.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:16 smithi150 ceph-mon[32761]: pgmap v153: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:27:16.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:16 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/2611213952' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T18:27:18.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:18 smithi115 ceph-mon[41073]: pgmap v154: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:18.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:18 smithi115 ceph-mon[29603]: pgmap v154: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:18.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:18 smithi150 ceph-mon[32761]: pgmap v154: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:19.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:27:18 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:27:18] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:27:19.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:19 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:27:19.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:19 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:27:19.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:19 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:27:20.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:20 smithi115 ceph-mon[41073]: pgmap v155: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:20.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:20 smithi115 ceph-mon[29603]: pgmap v155: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:20.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:20 smithi150 ceph-mon[32761]: pgmap v155: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:20.555 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:27:20 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:27:20.755 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.1 2024-09-10T18:27:21.068 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQCJjeBmUId4KxAAN986T3CaaFGHsvLfPCExdA== 2024-09-10T18:27:21.068 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQCJjeBmUId4KxAAN986T3CaaFGHsvLfPCExdA== == AQCJjeBmUId4KxAAN986T3CaaFGHsvLfPCExdA== ']' 2024-09-10T18:27:21.069 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:27:21.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:21 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:27:21.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:21 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/451490400' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T18:27:21.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:21 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:27:21.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:21 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/451490400' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T18:27:21.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:21 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:27:21.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:21 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/451490400' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T18:27:22.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:22 smithi115 ceph-mon[41073]: pgmap v156: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:27:22.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:22 smithi115 ceph-mon[29603]: pgmap v156: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:27:22.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:22 smithi150 ceph-mon[32761]: pgmap v156: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:27:24.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:24 smithi115 ceph-mon[41073]: pgmap v157: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:24.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:24 smithi115 ceph-mon[29603]: pgmap v157: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:24.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:24 smithi150 ceph-mon[32761]: pgmap v157: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:25.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:27:24 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:27:24] "GET /metrics HTTP/1.1" 200 219753 "" "Prometheus/2.43.0" 2024-09-10T18:27:26.070 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.1 2024-09-10T18:27:26.385 INFO:teuthology.orchestra.run.smithi115.stdout:rotating key for osd.2 2024-09-10T18:27:26.385 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQB5j+BmSbZsCxAAeVgCVQfZaKc7fBehcF6S9Q== 2024-09-10T18:27:26.385 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQCJjeBmUId4KxAAN986T3CaaFGHsvLfPCExdA== == AQB5j+BmSbZsCxAAeVgCVQfZaKc7fBehcF6S9Q== ']' 2024-09-10T18:27:26.385 INFO:teuthology.orchestra.run.smithi115.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-10T18:27:26.386 INFO:teuthology.orchestra.run.smithi115.stderr:+ echo 'rotating key for osd.2' 2024-09-10T18:27:26.386 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:26 smithi115 ceph-mon[41073]: pgmap v158: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:27:26.386 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:26 smithi115 ceph-mon[29603]: pgmap v158: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:27:26.386 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.2 2024-09-10T18:27:26.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:26 smithi150 ceph-mon[32761]: pgmap v158: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:27:26.687 INFO:teuthology.orchestra.run.smithi115.stderr:+ K=AQCijeBmeHwPExAA/jbfQdEAsD6ylb2WF1UKmQ== 2024-09-10T18:27:26.687 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQCijeBmeHwPExAA/jbfQdEAsD6ylb2WF1UKmQ== 2024-09-10T18:27:26.687 INFO:teuthology.orchestra.run.smithi115.stderr:+ ceph orch daemon rotate-key osd.2 2024-09-10T18:27:26.942 INFO:teuthology.orchestra.run.smithi115.stdout:Scheduled to rotate-key osd.2 on host 'smithi115' 2024-09-10T18:27:26.953 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQCijeBmeHwPExAA/jbfQdEAsD6ylb2WF1UKmQ== == AQCijeBmeHwPExAA/jbfQdEAsD6ylb2WF1UKmQ== ']' 2024-09-10T18:27:26.954 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:27:27.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:27 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1129173139' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T18:27:27.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:27 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/268327198' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T18:27:27.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:27 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:27.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:27 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:27.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:27 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:27:27.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:27 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1129173139' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T18:27:27.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:27 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/268327198' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T18:27:27.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:27 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:27.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:27 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:27.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:27 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:27:27.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:27 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1129173139' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T18:27:27.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:27 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/268327198' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T18:27:27.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:27 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:27.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:27 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:27.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:27 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:27:28.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:28 smithi115 ceph-mon[41073]: from='client.24947 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:27:28.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:28 smithi115 ceph-mon[41073]: Schedule rotate-key daemon osd.2 2024-09-10T18:27:28.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:28 smithi115 ceph-mon[41073]: pgmap v159: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:28.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:28 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:28.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:28 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:28.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:28 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:27:28.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:28 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:27:28.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:28 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:28.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:28 smithi115 ceph-mon[41073]: Rotating authentication key for osd.2 2024-09-10T18:27:28.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:28 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]: dispatch 2024-09-10T18:27:28.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:28 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]: dispatch 2024-09-10T18:27:28.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:28 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]': finished 2024-09-10T18:27:28.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:28 smithi115 ceph-mon[41073]: Reconfiguring daemon osd.2 on smithi115 2024-09-10T18:27:28.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:28 smithi115 ceph-mon[29603]: from='client.24947 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:27:28.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:28 smithi115 ceph-mon[29603]: Schedule rotate-key daemon osd.2 2024-09-10T18:27:28.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:28 smithi115 ceph-mon[29603]: pgmap v159: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:28.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:28 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:28.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:28 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:28.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:28 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:27:28.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:28 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:27:28.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:28 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:28.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:28 smithi115 ceph-mon[29603]: Rotating authentication key for osd.2 2024-09-10T18:27:28.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:28 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]: dispatch 2024-09-10T18:27:28.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:28 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]: dispatch 2024-09-10T18:27:28.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:28 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]': finished 2024-09-10T18:27:28.487 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:28 smithi115 ceph-mon[29603]: Reconfiguring daemon osd.2 on smithi115 2024-09-10T18:27:28.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:28 smithi150 ceph-mon[32761]: from='client.24947 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:27:28.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:28 smithi150 ceph-mon[32761]: Schedule rotate-key daemon osd.2 2024-09-10T18:27:28.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:28 smithi150 ceph-mon[32761]: pgmap v159: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:28.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:28 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:28.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:28 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:28.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:28 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:27:28.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:28 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:27:28.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:28 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:28.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:28 smithi150 ceph-mon[32761]: Rotating authentication key for osd.2 2024-09-10T18:27:28.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:28 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]: dispatch 2024-09-10T18:27:28.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:28 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]: dispatch 2024-09-10T18:27:28.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:28 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]': finished 2024-09-10T18:27:28.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:28 smithi150 ceph-mon[32761]: Reconfiguring daemon osd.2 on smithi115 2024-09-10T18:27:29.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:27:28 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:27:28] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:27:30.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:29 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:30.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:29 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:30.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:29 smithi115 ceph-mon[29603]: pgmap v160: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:30.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:29 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:30.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:29 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:30.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:29 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:30.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:29 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:30.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:29 smithi115 ceph-mon[41073]: pgmap v160: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:30.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:29 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:30.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:29 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:30.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:29 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:30.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:29 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:30.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:29 smithi150 ceph-mon[32761]: pgmap v160: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:30.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:29 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:30.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:29 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:30.805 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:27:30 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:27:31.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:30 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:27:31.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:30 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:27:31.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:30 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:27:31.956 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.2 2024-09-10T18:27:32.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:31 smithi115 ceph-mon[41073]: pgmap v161: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:27:32.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:31 smithi115 ceph-mon[29603]: pgmap v161: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:27:32.273 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQCijeBmeHwPExAA/jbfQdEAsD6ylb2WF1UKmQ== 2024-09-10T18:27:32.274 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQCijeBmeHwPExAA/jbfQdEAsD6ylb2WF1UKmQ== == AQCijeBmeHwPExAA/jbfQdEAsD6ylb2WF1UKmQ== ']' 2024-09-10T18:27:32.274 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:27:32.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:31 smithi150 ceph-mon[32761]: pgmap v161: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:27:33.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:32 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/330803640' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T18:27:33.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:32 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/330803640' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T18:27:33.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:32 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/330803640' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T18:27:34.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:33 smithi115 ceph-mon[41073]: pgmap v162: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:34.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:33 smithi115 ceph-mon[29603]: pgmap v162: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:34.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:33 smithi150 ceph-mon[32761]: pgmap v162: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:35.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:34 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:27:35.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:34 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:27:35.236 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:27:34 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:27:34] "GET /metrics HTTP/1.1" 200 219772 "" "Prometheus/2.43.0" 2024-09-10T18:27:35.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:34 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:27:36.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:35 smithi115 ceph-mon[41073]: pgmap v163: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:27:36.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:35 smithi115 ceph-mon[29603]: pgmap v163: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:27:36.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:35 smithi150 ceph-mon[32761]: pgmap v163: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:27:37.275 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.2 2024-09-10T18:27:37.582 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQCijeBmeHwPExAA/jbfQdEAsD6ylb2WF1UKmQ== 2024-09-10T18:27:37.582 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQCijeBmeHwPExAA/jbfQdEAsD6ylb2WF1UKmQ== == AQCijeBmeHwPExAA/jbfQdEAsD6ylb2WF1UKmQ== ']' 2024-09-10T18:27:37.582 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:27:38.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:38 smithi115 ceph-mon[41073]: pgmap v164: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:38.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:38 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1629637334' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T18:27:38.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:38 smithi115 ceph-mon[29603]: pgmap v164: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:38.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:38 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1629637334' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T18:27:38.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:38 smithi150 ceph-mon[32761]: pgmap v164: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:38.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:38 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1629637334' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T18:27:39.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:27:38 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:27:38] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:27:40.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:40 smithi115 ceph-mon[41073]: pgmap v165: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:40.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:40 smithi115 ceph-mon[29603]: pgmap v165: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:40.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:40 smithi150 ceph-mon[32761]: pgmap v165: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:40.555 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:27:40 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:27:41.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:41 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:27:41.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:41 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:27:41.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:41 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:27:42.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:42 smithi115 ceph-mon[41073]: pgmap v166: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:27:42.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:42 smithi115 ceph-mon[29603]: pgmap v166: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:27:42.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:42 smithi150 ceph-mon[32761]: pgmap v166: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:27:42.584 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.2 2024-09-10T18:27:42.895 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQCijeBmeHwPExAA/jbfQdEAsD6ylb2WF1UKmQ== 2024-09-10T18:27:42.896 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQCijeBmeHwPExAA/jbfQdEAsD6ylb2WF1UKmQ== == AQCijeBmeHwPExAA/jbfQdEAsD6ylb2WF1UKmQ== ']' 2024-09-10T18:27:42.896 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:27:43.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:43 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1861156113' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T18:27:43.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:43 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1861156113' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T18:27:43.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:43 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1861156113' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T18:27:44.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:44 smithi115 ceph-mon[41073]: pgmap v167: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:44.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:44 smithi115 ceph-mon[29603]: pgmap v167: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:44.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:44 smithi150 ceph-mon[32761]: pgmap v167: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:45.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:27:44 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:27:44] "GET /metrics HTTP/1.1" 200 219772 "" "Prometheus/2.43.0" 2024-09-10T18:27:45.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:45 smithi115 ceph-mon[41073]: pgmap v168: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:27:45.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:45 smithi115 ceph-mon[29603]: pgmap v168: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:27:46.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:45 smithi150 ceph-mon[32761]: pgmap v168: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:27:47.897 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.2 2024-09-10T18:27:48.211 INFO:teuthology.orchestra.run.smithi115.stdout:rotating key for osd.3 2024-09-10T18:27:48.212 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQCPj+Bm8PMoEhAAkir7XAEz2ksvfm+tEBkbVg== 2024-09-10T18:27:48.212 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQCijeBmeHwPExAA/jbfQdEAsD6ylb2WF1UKmQ== == AQCPj+Bm8PMoEhAAkir7XAEz2ksvfm+tEBkbVg== ']' 2024-09-10T18:27:48.212 INFO:teuthology.orchestra.run.smithi115.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-10T18:27:48.212 INFO:teuthology.orchestra.run.smithi115.stderr:+ echo 'rotating key for osd.3' 2024-09-10T18:27:48.212 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.3 2024-09-10T18:27:48.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:48 smithi115 ceph-mon[41073]: pgmap v169: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:48.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:48 smithi115 ceph-mon[29603]: pgmap v169: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:48.515 INFO:teuthology.orchestra.run.smithi115.stderr:+ K=AQC6jeBmEv8ZLxAAM1xtMgUxeGiXMkrHf4bCLQ== 2024-09-10T18:27:48.515 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQC6jeBmEv8ZLxAAM1xtMgUxeGiXMkrHf4bCLQ== 2024-09-10T18:27:48.515 INFO:teuthology.orchestra.run.smithi115.stderr:+ ceph orch daemon rotate-key osd.3 2024-09-10T18:27:48.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:48 smithi150 ceph-mon[32761]: pgmap v169: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:48.774 INFO:teuthology.orchestra.run.smithi115.stdout:Scheduled to rotate-key osd.3 on host 'smithi115' 2024-09-10T18:27:48.785 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQC6jeBmEv8ZLxAAM1xtMgUxeGiXMkrHf4bCLQ== == AQC6jeBmEv8ZLxAAM1xtMgUxeGiXMkrHf4bCLQ== ']' 2024-09-10T18:27:48.785 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:27:49.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:27:48 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:27:48] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:27:49.356 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:49 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1104539082' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T18:27:49.356 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:49 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/133748248' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T18:27:49.356 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:49 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:49.356 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:49 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:49.357 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:49 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:27:49.357 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:49 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:49.357 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:49 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:49.357 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:49 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:27:49.357 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:49 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:27:49.357 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:49 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:49.357 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:49 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:27:49.357 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:49 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]: dispatch 2024-09-10T18:27:49.357 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:49 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]: dispatch 2024-09-10T18:27:49.357 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:49 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]': finished 2024-09-10T18:27:49.357 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:49 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1104539082' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T18:27:49.357 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:49 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/133748248' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T18:27:49.358 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:49 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:49.358 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:49 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:49.358 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:49 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:27:49.358 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:49 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:49.358 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:49 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:49.358 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:49 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:27:49.358 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:49 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:27:49.358 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:49 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:49.358 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:49 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:27:49.358 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:49 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]: dispatch 2024-09-10T18:27:49.358 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:49 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]: dispatch 2024-09-10T18:27:49.358 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:49 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]': finished 2024-09-10T18:27:49.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:49 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1104539082' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T18:27:49.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:49 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/133748248' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T18:27:49.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:49 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:49.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:49 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:49.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:49 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:27:49.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:49 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:49.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:49 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:49.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:49 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:27:49.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:49 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:27:49.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:49 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:49.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:49 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:27:49.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:49 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]: dispatch 2024-09-10T18:27:49.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:49 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]: dispatch 2024-09-10T18:27:49.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:49 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]': finished 2024-09-10T18:27:50.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:50 smithi115 ceph-mon[29603]: from='client.24995 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:27:50.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:50 smithi115 ceph-mon[29603]: Schedule rotate-key daemon osd.3 2024-09-10T18:27:50.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:50 smithi115 ceph-mon[29603]: pgmap v170: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:50.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:50 smithi115 ceph-mon[29603]: Rotating authentication key for osd.3 2024-09-10T18:27:50.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:50 smithi115 ceph-mon[29603]: Reconfiguring daemon osd.3 on smithi115 2024-09-10T18:27:50.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:50 smithi115 ceph-mon[41073]: from='client.24995 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:27:50.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:50 smithi115 ceph-mon[41073]: Schedule rotate-key daemon osd.3 2024-09-10T18:27:50.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:50 smithi115 ceph-mon[41073]: pgmap v170: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:50.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:50 smithi115 ceph-mon[41073]: Rotating authentication key for osd.3 2024-09-10T18:27:50.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:50 smithi115 ceph-mon[41073]: Reconfiguring daemon osd.3 on smithi115 2024-09-10T18:27:50.555 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:27:50 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:27:50.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:50 smithi150 ceph-mon[32761]: from='client.24995 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:27:50.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:50 smithi150 ceph-mon[32761]: Schedule rotate-key daemon osd.3 2024-09-10T18:27:50.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:50 smithi150 ceph-mon[32761]: pgmap v170: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:50.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:50 smithi150 ceph-mon[32761]: Rotating authentication key for osd.3 2024-09-10T18:27:50.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:50 smithi150 ceph-mon[32761]: Reconfiguring daemon osd.3 on smithi115 2024-09-10T18:27:51.488 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:51 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:27:51.489 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:51 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:51.489 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:51 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:51.489 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:51 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:51.489 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:51 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:51.489 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:51 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:27:51.489 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:51 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:51.489 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:51 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:51.489 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:51 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:51.489 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:51 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:51.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:51 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:27:51.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:51 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:51.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:51 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:51.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:51 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:51.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:51 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:27:52.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:52 smithi115 ceph-mon[41073]: pgmap v171: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:27:52.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:52 smithi115 ceph-mon[29603]: pgmap v171: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:27:52.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:52 smithi150 ceph-mon[32761]: pgmap v171: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:27:53.788 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.3 2024-09-10T18:27:54.099 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQC6jeBmEv8ZLxAAM1xtMgUxeGiXMkrHf4bCLQ== 2024-09-10T18:27:54.099 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQC6jeBmEv8ZLxAAM1xtMgUxeGiXMkrHf4bCLQ== == AQC6jeBmEv8ZLxAAM1xtMgUxeGiXMkrHf4bCLQ== ']' 2024-09-10T18:27:54.099 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:27:54.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:54 smithi115 ceph-mon[41073]: pgmap v172: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:54.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:54 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1040431506' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T18:27:54.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:54 smithi115 ceph-mon[29603]: pgmap v172: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:54.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:54 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1040431506' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T18:27:54.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:54 smithi150 ceph-mon[32761]: pgmap v172: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:54.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:54 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1040431506' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T18:27:55.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:27:54 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:27:54] "GET /metrics HTTP/1.1" 200 219790 "" "Prometheus/2.43.0" 2024-09-10T18:27:56.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:56 smithi115 ceph-mon[41073]: pgmap v173: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:27:56.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:56 smithi115 ceph-mon[29603]: pgmap v173: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:27:56.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:56 smithi150 ceph-mon[32761]: pgmap v173: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:27:58.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:27:58 smithi115 ceph-mon[29603]: pgmap v174: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:58.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:27:58 smithi115 ceph-mon[41073]: pgmap v174: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:58.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:27:58 smithi150 ceph-mon[32761]: pgmap v174: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:27:59.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:27:58 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:27:58] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:27:59.102 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.3 2024-09-10T18:27:59.413 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQC6jeBmEv8ZLxAAM1xtMgUxeGiXMkrHf4bCLQ== 2024-09-10T18:27:59.413 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQC6jeBmEv8ZLxAAM1xtMgUxeGiXMkrHf4bCLQ== == AQC6jeBmEv8ZLxAAM1xtMgUxeGiXMkrHf4bCLQ== ']' 2024-09-10T18:27:59.413 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:28:00.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:00 smithi115 ceph-mon[41073]: pgmap v175: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:00.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:00 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/2395971775' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T18:28:00.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:00 smithi115 ceph-mon[29603]: pgmap v175: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:00.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:00 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/2395971775' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T18:28:00.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:00 smithi150 ceph-mon[32761]: pgmap v175: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:00.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:00 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/2395971775' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T18:28:00.555 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:28:00 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:28:01.456 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:01 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:28:01.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:01 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:28:01.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:01 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:28:02.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:02 smithi115 ceph-mon[41073]: pgmap v176: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:28:02.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:02 smithi115 ceph-mon[29603]: pgmap v176: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:28:02.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:02 smithi150 ceph-mon[32761]: pgmap v176: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:28:04.414 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.3 2024-09-10T18:28:04.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:04 smithi115 ceph-mon[41073]: pgmap v177: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:04.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:04 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:28:04.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:04 smithi115 ceph-mon[29603]: pgmap v177: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:04.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:04 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:28:04.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:04 smithi150 ceph-mon[32761]: pgmap v177: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:04.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:04 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:28:04.727 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQC6jeBmEv8ZLxAAM1xtMgUxeGiXMkrHf4bCLQ== 2024-09-10T18:28:04.727 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQC6jeBmEv8ZLxAAM1xtMgUxeGiXMkrHf4bCLQ== == AQC6jeBmEv8ZLxAAM1xtMgUxeGiXMkrHf4bCLQ== ']' 2024-09-10T18:28:04.728 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:28:04.985 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:28:04 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:28:04] "GET /metrics HTTP/1.1" 200 219792 "" "Prometheus/2.43.0" 2024-09-10T18:28:05.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:05 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/2237566750' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T18:28:05.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:05 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/2237566750' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T18:28:05.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:05 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/2237566750' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T18:28:06.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:06 smithi115 ceph-mon[29603]: pgmap v178: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:28:06.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:06 smithi115 ceph-mon[41073]: pgmap v178: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:28:06.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:06 smithi150 ceph-mon[32761]: pgmap v178: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:28:08.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:08 smithi115 ceph-mon[41073]: pgmap v179: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:08.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:08 smithi115 ceph-mon[29603]: pgmap v179: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:08.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:08 smithi150 ceph-mon[32761]: pgmap v179: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:09.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:28:08 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:28:08] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:28:09.729 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.3 2024-09-10T18:28:10.040 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQC6jeBmEv8ZLxAAM1xtMgUxeGiXMkrHf4bCLQ== 2024-09-10T18:28:10.040 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQC6jeBmEv8ZLxAAM1xtMgUxeGiXMkrHf4bCLQ== == AQC6jeBmEv8ZLxAAM1xtMgUxeGiXMkrHf4bCLQ== ']' 2024-09-10T18:28:10.040 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:28:10.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:10 smithi115 ceph-mon[41073]: pgmap v180: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:10.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:10 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/3421671457' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T18:28:10.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:10 smithi115 ceph-mon[29603]: pgmap v180: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:10.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:10 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/3421671457' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T18:28:10.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:10 smithi150 ceph-mon[32761]: pgmap v180: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:10.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:10 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/3421671457' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T18:28:10.555 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:28:10 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:28:11.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:11 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:28:11.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:11 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:28:11.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:11 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:28:12.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:12 smithi115 ceph-mon[41073]: pgmap v181: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:28:12.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:12 smithi115 ceph-mon[29603]: pgmap v181: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:28:12.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:12 smithi150 ceph-mon[32761]: pgmap v181: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:28:14.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:14 smithi115 ceph-mon[41073]: pgmap v182: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:14.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:14 smithi115 ceph-mon[29603]: pgmap v182: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:14.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:14 smithi150 ceph-mon[32761]: pgmap v182: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:15.042 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.3 2024-09-10T18:28:15.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:28:14 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:28:14] "GET /metrics HTTP/1.1" 200 219792 "" "Prometheus/2.43.0" 2024-09-10T18:28:15.355 INFO:teuthology.orchestra.run.smithi115.stdout:rotating key for osd.4 2024-09-10T18:28:15.355 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQClj+Bm/yZuCRAAMMfAmtNS/i5zO3tHsvLJcQ== 2024-09-10T18:28:15.356 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQC6jeBmEv8ZLxAAM1xtMgUxeGiXMkrHf4bCLQ== == AQClj+Bm/yZuCRAAMMfAmtNS/i5zO3tHsvLJcQ== ']' 2024-09-10T18:28:15.356 INFO:teuthology.orchestra.run.smithi115.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-10T18:28:15.356 INFO:teuthology.orchestra.run.smithi115.stderr:+ echo 'rotating key for osd.4' 2024-09-10T18:28:15.356 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.4 2024-09-10T18:28:15.658 INFO:teuthology.orchestra.run.smithi115.stderr:+ K=AQDRjeBm9doeBBAAWZNO5AIsrpdnhlzpY4PoSw== 2024-09-10T18:28:15.658 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQDRjeBm9doeBBAAWZNO5AIsrpdnhlzpY4PoSw== 2024-09-10T18:28:15.658 INFO:teuthology.orchestra.run.smithi115.stderr:+ ceph orch daemon rotate-key osd.4 2024-09-10T18:28:15.915 INFO:teuthology.orchestra.run.smithi115.stdout:Scheduled to rotate-key osd.4 on host 'smithi150' 2024-09-10T18:28:15.926 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQDRjeBm9doeBBAAWZNO5AIsrpdnhlzpY4PoSw== == AQDRjeBm9doeBBAAWZNO5AIsrpdnhlzpY4PoSw== ']' 2024-09-10T18:28:15.926 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:28:16.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:16 smithi115 ceph-mon[41073]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:28:16.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:16 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/3319116962' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T18:28:16.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:16 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/2421388703' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T18:28:16.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:16 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:16.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:16 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:16.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:16 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:28:16.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:16 smithi115 ceph-mon[29603]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:28:16.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:16 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/3319116962' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T18:28:16.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:16 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/2421388703' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T18:28:16.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:16 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:16.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:16 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:16.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:16 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:28:16.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:16 smithi150 ceph-mon[32761]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:28:16.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:16 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/3319116962' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T18:28:16.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:16 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/2421388703' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T18:28:16.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:16 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:16.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:16 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:16.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:16 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:28:17.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:17 smithi150 ceph-mon[32761]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:28:17.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:17 smithi150 ceph-mon[32761]: Schedule rotate-key daemon osd.4 2024-09-10T18:28:17.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:17 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:28:17.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:17 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:28:17.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:17 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:17.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:17 smithi150 ceph-mon[32761]: Rotating authentication key for osd.4 2024-09-10T18:28:17.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:17 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]: dispatch 2024-09-10T18:28:17.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:17 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]: dispatch 2024-09-10T18:28:17.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:17 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]': finished 2024-09-10T18:28:17.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:17 smithi150 ceph-mon[32761]: Reconfiguring daemon osd.4 on smithi150 2024-09-10T18:28:17.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:17 smithi115 ceph-mon[41073]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:28:17.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:17 smithi115 ceph-mon[41073]: Schedule rotate-key daemon osd.4 2024-09-10T18:28:17.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:17 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:28:17.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:17 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:28:17.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:17 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:17.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:17 smithi115 ceph-mon[41073]: Rotating authentication key for osd.4 2024-09-10T18:28:17.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:17 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]: dispatch 2024-09-10T18:28:17.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:17 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]: dispatch 2024-09-10T18:28:17.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:17 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]': finished 2024-09-10T18:28:17.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:17 smithi115 ceph-mon[41073]: Reconfiguring daemon osd.4 on smithi150 2024-09-10T18:28:17.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:17 smithi115 ceph-mon[29603]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:28:17.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:17 smithi115 ceph-mon[29603]: Schedule rotate-key daemon osd.4 2024-09-10T18:28:17.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:17 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:28:17.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:17 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:28:17.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:17 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:17.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:17 smithi115 ceph-mon[29603]: Rotating authentication key for osd.4 2024-09-10T18:28:17.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:17 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]: dispatch 2024-09-10T18:28:17.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:17 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]: dispatch 2024-09-10T18:28:17.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:17 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]': finished 2024-09-10T18:28:17.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:17 smithi115 ceph-mon[29603]: Reconfiguring daemon osd.4 on smithi150 2024-09-10T18:28:18.636 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:18 smithi115 ceph-mon[41073]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:18.636 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:18 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:18.636 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:18 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:18.636 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:18 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:18.636 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:18 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:18.637 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:18 smithi115 ceph-mon[29603]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:18.637 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:18 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:18.637 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:18 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:18.637 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:18 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:18.637 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:18 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:18.656 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:18 smithi150 ceph-mon[32761]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:18.656 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:18 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:18.656 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:18 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:18.656 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:18 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:18.656 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:18 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:19.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:28:18 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:28:18] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:28:19.649 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:19 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:28:19.649 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:19 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:28:19.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:19 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:28:20.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:20 smithi115 ceph-mon[29603]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:20.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:20 smithi115 ceph-mon[41073]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:20.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:20 smithi150 ceph-mon[32761]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:20.805 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:28:20 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:28:20.928 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.4 2024-09-10T18:28:21.230 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQDRjeBm9doeBBAAWZNO5AIsrpdnhlzpY4PoSw== 2024-09-10T18:28:21.230 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQDRjeBm9doeBBAAWZNO5AIsrpdnhlzpY4PoSw== == AQDRjeBm9doeBBAAWZNO5AIsrpdnhlzpY4PoSw== ']' 2024-09-10T18:28:21.230 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:28:21.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:21 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:28:21.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:21 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/2522079561' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T18:28:21.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:21 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:28:21.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:21 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/2522079561' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T18:28:21.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:21 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:28:21.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:21 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/2522079561' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T18:28:22.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:22 smithi115 ceph-mon[41073]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:28:22.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:22 smithi115 ceph-mon[29603]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:28:22.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:22 smithi150 ceph-mon[32761]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:28:24.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:24 smithi115 ceph-mon[41073]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:24.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:24 smithi115 ceph-mon[29603]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:24.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:24 smithi150 ceph-mon[32761]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:25.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:28:24 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:28:24] "GET /metrics HTTP/1.1" 200 219792 "" "Prometheus/2.43.0" 2024-09-10T18:28:26.231 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.4 2024-09-10T18:28:26.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:26 smithi115 ceph-mon[41073]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:28:26.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:26 smithi115 ceph-mon[29603]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:28:26.546 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQDRjeBm9doeBBAAWZNO5AIsrpdnhlzpY4PoSw== 2024-09-10T18:28:26.546 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQDRjeBm9doeBBAAWZNO5AIsrpdnhlzpY4PoSw== == AQDRjeBm9doeBBAAWZNO5AIsrpdnhlzpY4PoSw== ']' 2024-09-10T18:28:26.546 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:28:26.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:26 smithi150 ceph-mon[32761]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:28:27.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:27 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/2367918602' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T18:28:27.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:27 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/2367918602' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T18:28:27.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:27 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/2367918602' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T18:28:28.655 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:28 smithi150 ceph-mon[32761]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:28.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:28 smithi115 ceph-mon[41073]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:28.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:28 smithi115 ceph-mon[29603]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:29.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:28:28 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:28:28] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:28:30.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:30 smithi115 ceph-mon[41073]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:30.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:30 smithi115 ceph-mon[29603]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:30.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:30 smithi150 ceph-mon[32761]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:30.805 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:28:30 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:28:31.548 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.4 2024-09-10T18:28:31.706 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:31 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:28:31.734 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:31 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:28:31.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:31 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:28:31.862 INFO:teuthology.orchestra.run.smithi115.stdout:rotating key for osd.5 2024-09-10T18:28:31.863 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQDAj+Bm1tDxFRAAXOmKyUUJP9bOS1+/TAwP+w== 2024-09-10T18:28:31.863 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQDRjeBm9doeBBAAWZNO5AIsrpdnhlzpY4PoSw== == AQDAj+Bm1tDxFRAAXOmKyUUJP9bOS1+/TAwP+w== ']' 2024-09-10T18:28:31.863 INFO:teuthology.orchestra.run.smithi115.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-10T18:28:31.863 INFO:teuthology.orchestra.run.smithi115.stderr:+ echo 'rotating key for osd.5' 2024-09-10T18:28:31.863 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.5 2024-09-10T18:28:32.162 INFO:teuthology.orchestra.run.smithi115.stderr:+ K=AQDojeBmmnRhMBAAjURTgRGz17mwaXSykDc8gw== 2024-09-10T18:28:32.162 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQDojeBmmnRhMBAAjURTgRGz17mwaXSykDc8gw== 2024-09-10T18:28:32.162 INFO:teuthology.orchestra.run.smithi115.stderr:+ ceph orch daemon rotate-key osd.5 2024-09-10T18:28:32.418 INFO:teuthology.orchestra.run.smithi115.stdout:Scheduled to rotate-key osd.5 on host 'smithi150' 2024-09-10T18:28:32.419 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:32 smithi115 ceph-mon[41073]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:28:32.419 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:32 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/4275352604' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T18:28:32.419 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:32 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/21407349' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T18:28:32.419 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:32 smithi115 ceph-mon[29603]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:28:32.419 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:32 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/4275352604' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T18:28:32.419 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:32 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/21407349' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T18:28:32.431 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQDojeBmmnRhMBAAjURTgRGz17mwaXSykDc8gw== == AQDojeBmmnRhMBAAjURTgRGz17mwaXSykDc8gw== ']' 2024-09-10T18:28:32.431 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:28:32.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:32 smithi150 ceph-mon[32761]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:28:32.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:32 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/4275352604' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T18:28:32.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:32 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/21407349' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T18:28:33.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:33 smithi115 ceph-mon[41073]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.5", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:28:33.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:33 smithi115 ceph-mon[41073]: Schedule rotate-key daemon osd.5 2024-09-10T18:28:33.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:33 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:33.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:33 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:33.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:33 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:28:33.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:33 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:28:33.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:33 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:28:33.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:33 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:33.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:33 smithi115 ceph-mon[41073]: Rotating authentication key for osd.5 2024-09-10T18:28:33.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:33 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]: dispatch 2024-09-10T18:28:33.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:33 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]: dispatch 2024-09-10T18:28:33.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:33 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]': finished 2024-09-10T18:28:33.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:33 smithi115 ceph-mon[41073]: Reconfiguring daemon osd.5 on smithi150 2024-09-10T18:28:33.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:33 smithi115 ceph-mon[29603]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.5", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:28:33.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:33 smithi115 ceph-mon[29603]: Schedule rotate-key daemon osd.5 2024-09-10T18:28:33.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:33 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:33.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:33 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:33.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:33 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:28:33.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:33 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:28:33.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:33 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:28:33.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:33 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:33.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:33 smithi115 ceph-mon[29603]: Rotating authentication key for osd.5 2024-09-10T18:28:33.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:33 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]: dispatch 2024-09-10T18:28:33.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:33 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]: dispatch 2024-09-10T18:28:33.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:33 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]': finished 2024-09-10T18:28:33.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:33 smithi115 ceph-mon[29603]: Reconfiguring daemon osd.5 on smithi150 2024-09-10T18:28:33.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:33 smithi150 ceph-mon[32761]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.5", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:28:33.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:33 smithi150 ceph-mon[32761]: Schedule rotate-key daemon osd.5 2024-09-10T18:28:33.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:33 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:33.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:33 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:33.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:33 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:28:33.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:33 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:28:33.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:33 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:28:33.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:33 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:33.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:33 smithi150 ceph-mon[32761]: Rotating authentication key for osd.5 2024-09-10T18:28:33.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:33 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]: dispatch 2024-09-10T18:28:33.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:33 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]: dispatch 2024-09-10T18:28:33.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:33 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]': finished 2024-09-10T18:28:33.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:33 smithi150 ceph-mon[32761]: Reconfiguring daemon osd.5 on smithi150 2024-09-10T18:28:34.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:34 smithi150 ceph-mon[32761]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:34.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:34 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:34.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:34 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:34.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:34 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:28:34.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:34 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:34.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:34 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:34.902 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:34 smithi115 ceph-mon[41073]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:34.902 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:34 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:34.902 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:34 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:34.902 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:34 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:28:34.902 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:34 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:34.902 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:34 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:34.903 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:34 smithi115 ceph-mon[29603]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:34.903 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:34 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:34.903 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:34 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:34.903 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:34 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:28:34.903 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:34 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:34.903 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:34 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:28:35.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:28:34 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:28:34] "GET /metrics HTTP/1.1" 200 219796 "" "Prometheus/2.43.0" 2024-09-10T18:28:36.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:36 smithi150 ceph-mon[32761]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:28:36.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:36 smithi115 ceph-mon[29603]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:28:36.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:36 smithi115 ceph-mon[41073]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:28:37.433 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.5 2024-09-10T18:28:37.750 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQDojeBmmnRhMBAAjURTgRGz17mwaXSykDc8gw== 2024-09-10T18:28:37.751 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQDojeBmmnRhMBAAjURTgRGz17mwaXSykDc8gw== == AQDojeBmmnRhMBAAjURTgRGz17mwaXSykDc8gw== ']' 2024-09-10T18:28:37.751 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:28:38.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:38 smithi150 ceph-mon[32761]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:38.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:38 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/469876996' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T18:28:38.805 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:28:38 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:28:38] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:28:38.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:38 smithi115 ceph-mon[29603]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:38.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:38 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/469876996' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T18:28:38.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:38 smithi115 ceph-mon[41073]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:38.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:38 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/469876996' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T18:28:40.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:40 smithi150 ceph-mon[32761]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:40.805 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:28:40 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:28:40.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:40 smithi115 ceph-mon[41073]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:40.986 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:40 smithi115 ceph-mon[29603]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:41.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:41 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:28:41.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:41 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:28:41.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:41 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:28:42.752 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.5 2024-09-10T18:28:42.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:42 smithi150 ceph-mon[32761]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:28:42.984 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:42 smithi115 ceph-mon[41073]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:28:42.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:42 smithi115 ceph-mon[29603]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:28:43.067 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQDojeBmmnRhMBAAjURTgRGz17mwaXSykDc8gw== 2024-09-10T18:28:43.067 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQDojeBmmnRhMBAAjURTgRGz17mwaXSykDc8gw== == AQDojeBmmnRhMBAAjURTgRGz17mwaXSykDc8gw== ']' 2024-09-10T18:28:43.067 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:28:43.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:43 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1152033958' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T18:28:43.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:43 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1152033958' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T18:28:43.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:43 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1152033958' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T18:28:44.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:44 smithi150 ceph-mon[32761]: pgmap v197: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:44.902 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:44 smithi115 ceph-mon[41073]: pgmap v197: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:44.902 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:44 smithi115 ceph-mon[29603]: pgmap v197: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:45.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:28:44 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:28:44] "GET /metrics HTTP/1.1" 200 219796 "" "Prometheus/2.43.0" 2024-09-10T18:28:46.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:46 smithi150 ceph-mon[32761]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:28:46.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:46 smithi115 ceph-mon[41073]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:28:46.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:46 smithi115 ceph-mon[29603]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:28:48.069 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.5 2024-09-10T18:28:48.385 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQDojeBmmnRhMBAAjURTgRGz17mwaXSykDc8gw== 2024-09-10T18:28:48.385 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQDojeBmmnRhMBAAjURTgRGz17mwaXSykDc8gw== == AQDojeBmmnRhMBAAjURTgRGz17mwaXSykDc8gw== ']' 2024-09-10T18:28:48.386 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:28:48.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:48 smithi115 ceph-mon[41073]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:48.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:48 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/532997783' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T18:28:48.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:48 smithi115 ceph-mon[29603]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:48.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:48 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/532997783' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T18:28:48.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:48 smithi150 ceph-mon[32761]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:48.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:48 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/532997783' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T18:28:48.805 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:28:48 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:28:48] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:28:49.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:49 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:28:49.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:49 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:28:49.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:49 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:28:50.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:50 smithi150 ceph-mon[32761]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:50.805 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:28:50 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:28:50.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:50 smithi115 ceph-mon[29603]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:50.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:50 smithi115 ceph-mon[41073]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:51.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:51 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:28:51.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:51 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:28:51.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:51 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:28:52.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:52 smithi115 ceph-mon[29603]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:28:52.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:52 smithi115 ceph-mon[41073]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:28:53.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:52 smithi150 ceph-mon[32761]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:28:53.387 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.5 2024-09-10T18:28:53.701 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQDojeBmmnRhMBAAjURTgRGz17mwaXSykDc8gw== 2024-09-10T18:28:53.701 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQDojeBmmnRhMBAAjURTgRGz17mwaXSykDc8gw== == AQDojeBmmnRhMBAAjURTgRGz17mwaXSykDc8gw== ']' 2024-09-10T18:28:53.701 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:28:54.901 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:54 smithi115 ceph-mon[41073]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:54.901 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:54 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/2526683888' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T18:28:54.901 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:54 smithi115 ceph-mon[29603]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:54.902 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:54 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/2526683888' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T18:28:55.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:54 smithi150 ceph-mon[32761]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:55.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:54 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/2526683888' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T18:28:55.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:28:54 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:28:54] "GET /metrics HTTP/1.1" 200 219800 "" "Prometheus/2.43.0" 2024-09-10T18:28:56.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:56 smithi115 ceph-mon[41073]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:28:56.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:56 smithi115 ceph-mon[29603]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:28:57.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:56 smithi150 ceph-mon[32761]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:28:58.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:57 smithi150 ceph-mon[32761]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:58.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:57 smithi115 ceph-mon[41073]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:58.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:57 smithi115 ceph-mon[29603]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:28:58.703 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.5 2024-09-10T18:28:59.017 INFO:teuthology.orchestra.run.smithi115.stdout:rotating key for osd.6 2024-09-10T18:28:59.017 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQDQj+Bm64F8HRAA5F1eoWrzEmjJqJnRoolZiQ== 2024-09-10T18:28:59.017 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQDojeBmmnRhMBAAjURTgRGz17mwaXSykDc8gw== == AQDQj+Bm64F8HRAA5F1eoWrzEmjJqJnRoolZiQ== ']' 2024-09-10T18:28:59.017 INFO:teuthology.orchestra.run.smithi115.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-10T18:28:59.018 INFO:teuthology.orchestra.run.smithi115.stderr:+ echo 'rotating key for osd.6' 2024-09-10T18:28:59.018 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.6 2024-09-10T18:28:59.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:28:58 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:28:58] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:28:59.320 INFO:teuthology.orchestra.run.smithi115.stderr:+ K=AQAAjuBmTmboIxAAQY6z20afY19EfEz8VJR57A== 2024-09-10T18:28:59.320 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAAjuBmTmboIxAAQY6z20afY19EfEz8VJR57A== 2024-09-10T18:28:59.321 INFO:teuthology.orchestra.run.smithi115.stderr:+ ceph orch daemon rotate-key osd.6 2024-09-10T18:28:59.321 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:28:59 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/2463394533' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T18:28:59.321 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:28:59 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/2463394533' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T18:28:59.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:28:59 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/2463394533' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T18:28:59.577 INFO:teuthology.orchestra.run.smithi115.stdout:Scheduled to rotate-key osd.6 on host 'smithi150' 2024-09-10T18:28:59.589 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAAjuBmTmboIxAAQY6z20afY19EfEz8VJR57A== == AQAAjuBmTmboIxAAQY6z20afY19EfEz8VJR57A== ']' 2024-09-10T18:28:59.589 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:29:00.289 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:00 smithi150 ceph-mon[32761]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:00.289 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:00 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/319304530' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T18:29:00.289 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:00 smithi150 ceph-mon[32761]: from='client.25145 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.6", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:29:00.289 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:00 smithi150 ceph-mon[32761]: Schedule rotate-key daemon osd.6 2024-09-10T18:29:00.289 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:00 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:00.290 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:00 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:00.290 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:00 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:29:00.290 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:00 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:29:00.290 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:00 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:29:00.290 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:00 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:00.290 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:00 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]: dispatch 2024-09-10T18:29:00.290 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:00 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]: dispatch 2024-09-10T18:29:00.290 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:00 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]': finished 2024-09-10T18:29:00.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:00 smithi115 ceph-mon[29603]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:00.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:00 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/319304530' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T18:29:00.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:00 smithi115 ceph-mon[29603]: from='client.25145 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.6", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:29:00.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:00 smithi115 ceph-mon[29603]: Schedule rotate-key daemon osd.6 2024-09-10T18:29:00.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:00 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:00.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:00 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:00.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:00 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:29:00.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:00 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:29:00.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:00 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:29:00.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:00 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:00.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:00 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]: dispatch 2024-09-10T18:29:00.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:00 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]: dispatch 2024-09-10T18:29:00.486 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:00 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]': finished 2024-09-10T18:29:00.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:00 smithi115 ceph-mon[41073]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:00.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:00 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/319304530' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T18:29:00.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:00 smithi115 ceph-mon[41073]: from='client.25145 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.6", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:29:00.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:00 smithi115 ceph-mon[41073]: Schedule rotate-key daemon osd.6 2024-09-10T18:29:00.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:00 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:00.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:00 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:00.487 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:00 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:29:00.487 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:00 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:29:00.487 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:00 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:29:00.487 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:00 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:00.487 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:00 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]: dispatch 2024-09-10T18:29:00.487 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:00 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]: dispatch 2024-09-10T18:29:00.487 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:00 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]': finished 2024-09-10T18:29:00.805 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:29:00 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:29:01.235 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:01 smithi150 ceph-mon[32761]: Rotating authentication key for osd.6 2024-09-10T18:29:01.235 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:01 smithi150 ceph-mon[32761]: Reconfiguring daemon osd.6 on smithi150 2024-09-10T18:29:01.235 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:01 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:29:01.484 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:01 smithi115 ceph-mon[29603]: Rotating authentication key for osd.6 2024-09-10T18:29:01.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:01 smithi115 ceph-mon[29603]: Reconfiguring daemon osd.6 on smithi150 2024-09-10T18:29:01.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:01 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:29:01.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:01 smithi115 ceph-mon[41073]: Rotating authentication key for osd.6 2024-09-10T18:29:01.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:01 smithi115 ceph-mon[41073]: Reconfiguring daemon osd.6 on smithi150 2024-09-10T18:29:01.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:01 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:29:02.578 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:02 smithi150 ceph-mon[32761]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:29:02.578 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:02 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:02.578 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:02 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:02.578 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:02 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:02.578 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:02 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:02.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:02 smithi115 ceph-mon[29603]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:29:02.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:02 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:02.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:02 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:02.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:02 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:02.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:02 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:02.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:02 smithi115 ceph-mon[41073]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:29:02.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:02 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:02.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:02 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:02.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:02 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:02.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:02 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:04.591 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.6 2024-09-10T18:29:04.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:04 smithi115 ceph-mon[41073]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:04.592 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:04 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:29:04.592 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:04 smithi115 ceph-mon[29603]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:04.592 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:04 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:29:04.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:04 smithi150 ceph-mon[32761]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:04.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:04 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:29:04.905 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAAjuBmTmboIxAAQY6z20afY19EfEz8VJR57A== 2024-09-10T18:29:04.905 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAAjuBmTmboIxAAQY6z20afY19EfEz8VJR57A== == AQAAjuBmTmboIxAAQY6z20afY19EfEz8VJR57A== ']' 2024-09-10T18:29:04.905 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:29:05.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:04 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:29:04] "GET /metrics HTTP/1.1" 200 219805 "" "Prometheus/2.43.0" 2024-09-10T18:29:05.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:05 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/3135471659' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T18:29:05.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:05 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/3135471659' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T18:29:05.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:05 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/3135471659' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T18:29:06.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:06 smithi115 ceph-mon[41073]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:29:06.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:06 smithi115 ceph-mon[29603]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:29:06.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:06 smithi150 ceph-mon[32761]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:29:08.656 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:08 smithi150 ceph-mon[32761]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:08.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:08 smithi115 ceph-mon[41073]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:08.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:08 smithi115 ceph-mon[29603]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:09.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:08 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:29:08] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:29:09.907 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.6 2024-09-10T18:29:10.223 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAAjuBmTmboIxAAQY6z20afY19EfEz8VJR57A== 2024-09-10T18:29:10.224 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAAjuBmTmboIxAAQY6z20afY19EfEz8VJR57A== == AQAAjuBmTmboIxAAQY6z20afY19EfEz8VJR57A== ']' 2024-09-10T18:29:10.224 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:29:10.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:10 smithi115 ceph-mon[41073]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:10.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:10 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/3952663797' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T18:29:10.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:10 smithi115 ceph-mon[29603]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:10.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:10 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/3952663797' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T18:29:10.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:10 smithi150 ceph-mon[32761]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:10.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:10 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/3952663797' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T18:29:10.805 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:29:10 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:29:11.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:11 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:29:11.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:11 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:29:11.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:11 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:29:12.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:12 smithi115 ceph-mon[29603]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:29:12.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:12 smithi115 ceph-mon[41073]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:29:12.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:12 smithi150 ceph-mon[32761]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:29:14.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:14 smithi115 ceph-mon[41073]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:14.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:14 smithi115 ceph-mon[29603]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:14.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:14 smithi150 ceph-mon[32761]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:15.225 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.6 2024-09-10T18:29:15.226 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:14 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:29:14] "GET /metrics HTTP/1.1" 200 219805 "" "Prometheus/2.43.0" 2024-09-10T18:29:15.539 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAAjuBmTmboIxAAQY6z20afY19EfEz8VJR57A== 2024-09-10T18:29:15.539 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAAjuBmTmboIxAAQY6z20afY19EfEz8VJR57A== == AQAAjuBmTmboIxAAQY6z20afY19EfEz8VJR57A== ']' 2024-09-10T18:29:15.539 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:29:16.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:16 smithi115 ceph-mon[29603]: pgmap v213: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:29:16.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:16 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/4096856583' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T18:29:16.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:16 smithi115 ceph-mon[41073]: pgmap v213: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:29:16.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:16 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/4096856583' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T18:29:16.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:16 smithi150 ceph-mon[32761]: pgmap v213: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:29:16.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:16 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/4096856583' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T18:29:18.655 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:18 smithi150 ceph-mon[32761]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:18.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:18 smithi115 ceph-mon[41073]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:18.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:18 smithi115 ceph-mon[29603]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:19.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:18 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:29:18] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:29:19.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:19 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:29:19.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:19 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:29:19.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:19 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:29:20.541 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.6 2024-09-10T18:29:20.734 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:20 smithi115 ceph-mon[41073]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:20.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:20 smithi115 ceph-mon[29603]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:20.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:20 smithi150 ceph-mon[32761]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:20.805 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:29:20 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:29:20.856 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAAjuBmTmboIxAAQY6z20afY19EfEz8VJR57A== 2024-09-10T18:29:20.856 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAAjuBmTmboIxAAQY6z20afY19EfEz8VJR57A== == AQAAjuBmTmboIxAAQY6z20afY19EfEz8VJR57A== ']' 2024-09-10T18:29:20.856 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:29:21.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:21 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:29:21.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:21 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/4278508964' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T18:29:21.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:21 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:29:21.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:21 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/4278508964' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T18:29:21.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:21 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:29:21.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:21 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/4278508964' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T18:29:22.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:22 smithi115 ceph-mon[41073]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:29:22.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:22 smithi115 ceph-mon[29603]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:29:22.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:22 smithi150 ceph-mon[32761]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:29:24.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:24 smithi115 ceph-mon[41073]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:24.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:24 smithi115 ceph-mon[29603]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:24.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:24 smithi150 ceph-mon[32761]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:25.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:24 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:29:24] "GET /metrics HTTP/1.1" 200 219806 "" "Prometheus/2.43.0" 2024-09-10T18:29:25.858 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.6 2024-09-10T18:29:26.171 INFO:teuthology.orchestra.run.smithi115.stdout:rotating key for osd.7 2024-09-10T18:29:26.172 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQDrj+BmKNJMJxAAoFo1duvGslnvBWNZeqSZOA== 2024-09-10T18:29:26.172 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAAjuBmTmboIxAAQY6z20afY19EfEz8VJR57A== == AQDrj+BmKNJMJxAAoFo1duvGslnvBWNZeqSZOA== ']' 2024-09-10T18:29:26.172 INFO:teuthology.orchestra.run.smithi115.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-10T18:29:26.172 INFO:teuthology.orchestra.run.smithi115.stderr:+ echo 'rotating key for osd.7' 2024-09-10T18:29:26.172 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.7 2024-09-10T18:29:26.478 INFO:teuthology.orchestra.run.smithi115.stderr:+ K=AQAYjuBmOGbaDRAA8WsTylXacBGSBkX1Rc4fqQ== 2024-09-10T18:29:26.478 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAYjuBmOGbaDRAA8WsTylXacBGSBkX1Rc4fqQ== 2024-09-10T18:29:26.478 INFO:teuthology.orchestra.run.smithi115.stderr:+ ceph orch daemon rotate-key osd.7 2024-09-10T18:29:26.479 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:26 smithi115 ceph-mon[41073]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:29:26.479 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:26 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1590563530' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T18:29:26.479 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:26 smithi115 ceph-mon[29603]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:29:26.479 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:26 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1590563530' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T18:29:26.734 INFO:teuthology.orchestra.run.smithi115.stdout:Scheduled to rotate-key osd.7 on host 'smithi150' 2024-09-10T18:29:26.748 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAYjuBmOGbaDRAA8WsTylXacBGSBkX1Rc4fqQ== == AQAYjuBmOGbaDRAA8WsTylXacBGSBkX1Rc4fqQ== ']' 2024-09-10T18:29:26.749 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:29:26.807 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:26 smithi150 ceph-mon[32761]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:29:26.807 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:26 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1590563530' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T18:29:27.628 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:27 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/616320425' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T18:29:27.628 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:27 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:27.628 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:27 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:27.628 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:27 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:29:27.628 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:27 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:29:27.628 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:27 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:29:27.628 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:27 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:27.628 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:27 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]: dispatch 2024-09-10T18:29:27.629 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:27 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]: dispatch 2024-09-10T18:29:27.629 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:27 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]': finished 2024-09-10T18:29:27.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:27 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/616320425' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T18:29:27.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:27 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:27.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:27 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:27.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:27 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:29:27.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:27 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:29:27.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:27 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:29:27.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:27 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:27.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:27 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]: dispatch 2024-09-10T18:29:27.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:27 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]: dispatch 2024-09-10T18:29:27.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:27 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]': finished 2024-09-10T18:29:27.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:27 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/616320425' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T18:29:27.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:27 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:27.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:27 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:27.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:27 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:29:27.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:27 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:29:27.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:27 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:29:27.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:27 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:27.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:27 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]: dispatch 2024-09-10T18:29:27.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:27 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]: dispatch 2024-09-10T18:29:27.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:27 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]': finished 2024-09-10T18:29:28.482 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:28 smithi150 ceph-mon[32761]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.7", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:29:28.482 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:28 smithi150 ceph-mon[32761]: Schedule rotate-key daemon osd.7 2024-09-10T18:29:28.482 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:28 smithi150 ceph-mon[32761]: pgmap v219: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:28.483 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:28 smithi150 ceph-mon[32761]: Rotating authentication key for osd.7 2024-09-10T18:29:28.483 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:28 smithi150 ceph-mon[32761]: Reconfiguring daemon osd.7 on smithi150 2024-09-10T18:29:28.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:28 smithi115 ceph-mon[41073]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.7", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:29:28.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:28 smithi115 ceph-mon[41073]: Schedule rotate-key daemon osd.7 2024-09-10T18:29:28.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:28 smithi115 ceph-mon[41073]: pgmap v219: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:28.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:28 smithi115 ceph-mon[41073]: Rotating authentication key for osd.7 2024-09-10T18:29:28.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:28 smithi115 ceph-mon[41073]: Reconfiguring daemon osd.7 on smithi150 2024-09-10T18:29:28.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:28 smithi115 ceph-mon[29603]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.7", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:29:28.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:28 smithi115 ceph-mon[29603]: Schedule rotate-key daemon osd.7 2024-09-10T18:29:28.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:28 smithi115 ceph-mon[29603]: pgmap v219: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:28.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:28 smithi115 ceph-mon[29603]: Rotating authentication key for osd.7 2024-09-10T18:29:28.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:28 smithi115 ceph-mon[29603]: Reconfiguring daemon osd.7 on smithi150 2024-09-10T18:29:28.805 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:28 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:29:28] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:29:29.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:29 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:29.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:29 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:29.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:29 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:29.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:29 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:29.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:29 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:29.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:29 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:29.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:29 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:29.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:29 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:30.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:29 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:30.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:29 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:30.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:29 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:30.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:29 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:30.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:30 smithi150 ceph-mon[32761]: pgmap v220: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:30.805 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:29:30 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:29:30.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:30 smithi115 ceph-mon[41073]: pgmap v220: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:30.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:30 smithi115 ceph-mon[29603]: pgmap v220: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:31.750 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.7 2024-09-10T18:29:31.984 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:31 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:29:31.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:31 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:29:32.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:31 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:29:32.061 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAYjuBmOGbaDRAA8WsTylXacBGSBkX1Rc4fqQ== 2024-09-10T18:29:32.061 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAYjuBmOGbaDRAA8WsTylXacBGSBkX1Rc4fqQ== == AQAYjuBmOGbaDRAA8WsTylXacBGSBkX1Rc4fqQ== ']' 2024-09-10T18:29:32.061 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:29:32.972 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:32 smithi150 ceph-mon[32761]: pgmap v221: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:29:32.972 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:32 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1766292792' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T18:29:32.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:32 smithi115 ceph-mon[41073]: pgmap v221: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:29:32.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:32 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1766292792' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T18:29:32.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:32 smithi115 ceph-mon[29603]: pgmap v221: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:29:32.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:32 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1766292792' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T18:29:34.901 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:34 smithi115 ceph-mon[41073]: pgmap v222: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:34.901 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:34 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:29:34.901 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:34 smithi115 ceph-mon[29603]: pgmap v222: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:34.902 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:34 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:29:35.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:34 smithi150 ceph-mon[32761]: pgmap v222: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:35.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:34 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:29:35.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:34 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:29:34] "GET /metrics HTTP/1.1" 200 219806 "" "Prometheus/2.43.0" 2024-09-10T18:29:36.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:36 smithi115 ceph-mon[29603]: pgmap v223: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:29:36.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:36 smithi115 ceph-mon[41073]: pgmap v223: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:29:37.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:36 smithi150 ceph-mon[32761]: pgmap v223: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:29:37.063 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.7 2024-09-10T18:29:37.373 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAYjuBmOGbaDRAA8WsTylXacBGSBkX1Rc4fqQ== 2024-09-10T18:29:37.373 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAYjuBmOGbaDRAA8WsTylXacBGSBkX1Rc4fqQ== == AQAYjuBmOGbaDRAA8WsTylXacBGSBkX1Rc4fqQ== ']' 2024-09-10T18:29:37.373 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:29:37.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:37 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1485795819' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T18:29:37.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:37 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1485795819' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T18:29:38.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:37 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1485795819' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T18:29:38.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:38 smithi115 ceph-mon[41073]: pgmap v224: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:38.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:38 smithi115 ceph-mon[29603]: pgmap v224: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:39.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:38 smithi150 ceph-mon[32761]: pgmap v224: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:39.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:38 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:29:38] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:29:40.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:40 smithi150 ceph-mon[32761]: pgmap v225: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:40.805 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:29:40 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:29:40.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:40 smithi115 ceph-mon[41073]: pgmap v225: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:40.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:40 smithi115 ceph-mon[29603]: pgmap v225: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:41.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:41 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:29:41.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:41 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:29:42.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:41 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:29:42.374 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.7 2024-09-10T18:29:42.687 INFO:teuthology.orchestra.run.smithi115.stdout:rotating key for mgr.y 2024-09-10T18:29:42.688 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAHkOBmeFbSChAAigDJiRgqaQOPYcFZt6rBAg== 2024-09-10T18:29:42.688 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAYjuBmOGbaDRAA8WsTylXacBGSBkX1Rc4fqQ== == AQAHkOBmeFbSChAAigDJiRgqaQOPYcFZt6rBAg== ']' 2024-09-10T18:29:42.688 INFO:teuthology.orchestra.run.smithi115.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-10T18:29:42.688 INFO:teuthology.orchestra.run.smithi115.stderr:+ echo 'rotating key for mgr.y' 2024-09-10T18:29:42.688 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:42 smithi115 ceph-mon[41073]: pgmap v226: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:29:42.688 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:42 smithi115 ceph-mon[29603]: pgmap v226: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:29:42.689 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key mgr.y 2024-09-10T18:29:42.990 INFO:teuthology.orchestra.run.smithi115.stderr:+ K=AQD2jOBmmh07NBAAr+VG9WQKWtbLqgijPlotBw== 2024-09-10T18:29:42.990 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQD2jOBmmh07NBAAr+VG9WQKWtbLqgijPlotBw== 2024-09-10T18:29:42.990 INFO:teuthology.orchestra.run.smithi115.stderr:+ ceph orch daemon rotate-key mgr.y 2024-09-10T18:29:43.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:42 smithi150 ceph-mon[32761]: pgmap v226: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:29:43.249 INFO:teuthology.orchestra.run.smithi115.stdout:Scheduled to rotate-key mgr.y on host 'smithi115' 2024-09-10T18:29:43.260 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQD2jOBmmh07NBAAr+VG9WQKWtbLqgijPlotBw== == AQD2jOBmmh07NBAAr+VG9WQKWtbLqgijPlotBw== ']' 2024-09-10T18:29:43.260 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:29:43.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:43 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1978935323' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T18:29:43.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:43 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/175640494' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-10T18:29:43.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:43 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:43.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:43 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:43.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:43 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:29:43.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:43 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:29:43.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:43 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:29:43.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:43 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:43.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:43 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-10T18:29:43.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:43 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-10T18:29:43.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:43 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-10T18:29:43.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:43 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:29:43.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:43 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:29:43.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:43 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:29:43.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:43 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:29:43.959 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:43 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1978935323' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T18:29:43.959 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:43 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/175640494' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-10T18:29:43.959 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:43 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:43.959 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:43 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:43.959 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:43 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:29:43.959 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:43 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:29:43.959 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:43 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:29:43.959 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:43 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:43.959 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:43 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-10T18:29:43.959 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:43 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-10T18:29:43.959 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:43 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-10T18:29:43.960 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:43 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:29:43.960 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:43 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:29:43.960 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:43 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:29:43.960 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:43 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:29:43.960 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:43 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1978935323' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T18:29:43.960 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:43 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/175640494' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-10T18:29:43.960 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:43 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:43.960 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:43 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:43.960 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:43 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:29:43.960 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:43 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:29:43.960 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:43 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:29:43.961 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:43 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:43.961 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:43 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-10T18:29:43.961 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:43 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-10T18:29:43.961 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:43 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-10T18:29:43.961 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:43 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:29:43.961 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:43 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:29:43.961 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:43 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:29:43.961 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:43 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:29:44.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:44 smithi115 ceph-mon[41073]: pgmap v227: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:44.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:44 smithi115 ceph-mon[41073]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.y", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:29:44.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:44 smithi115 ceph-mon[41073]: Schedule rotate-key daemon mgr.y 2024-09-10T18:29:44.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:44 smithi115 ceph-mon[41073]: Rotating authentication key for mgr.y 2024-09-10T18:29:44.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:44 smithi115 ceph-mon[41073]: Reconfiguring daemon mgr.y on smithi115 2024-09-10T18:29:44.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:44 smithi115 ceph-mon[29603]: pgmap v227: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:44.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:44 smithi115 ceph-mon[29603]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.y", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:29:44.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:44 smithi115 ceph-mon[29603]: Schedule rotate-key daemon mgr.y 2024-09-10T18:29:44.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:44 smithi115 ceph-mon[29603]: Rotating authentication key for mgr.y 2024-09-10T18:29:44.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:44 smithi115 ceph-mon[29603]: Reconfiguring daemon mgr.y on smithi115 2024-09-10T18:29:45.028 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:44 smithi150 ceph-mon[32761]: pgmap v227: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:29:45.028 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:44 smithi150 ceph-mon[32761]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.y", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:29:45.028 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:44 smithi150 ceph-mon[32761]: Schedule rotate-key daemon mgr.y 2024-09-10T18:29:45.028 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:44 smithi150 ceph-mon[32761]: Rotating authentication key for mgr.y 2024-09-10T18:29:45.028 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:44 smithi150 ceph-mon[32761]: Reconfiguring daemon mgr.y on smithi115 2024-09-10T18:29:45.154 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:44 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:29:44] "GET /metrics HTTP/1.1" 200 219806 "" "Prometheus/2.43.0" 2024-09-10T18:29:45.154 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.004+0000 7faea6c58640 -1 mgr.server reply reply (13) Permission denied access denied: does your client key have mgr caps? See http://docs.ceph.com/en/latest/mgr/administrator/#client-authentication 2024-09-10T18:29:45.154 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.006+0000 7fae98bbc640 -1 log_channel(cephadm) log [ERR] : Non-zero return from ['ceph', '-k', '/var/lib/ceph/mgr/ceph-y/keyring', '-n', 'mgr.y', 'tell', 'mgr.y', 'rotate-key', '-i', '-']: 2024-09-10T18:29:44.999+0000 7fd90a7fc640 1 -- 172.21.15.115:0/1610273572 <== mon.0 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x7fd8f8003710 con 0x7fd90c104710 2024-09-10T18:29:45.154 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:44.999+0000 7fd90a7fc640 0 cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 1648168043697291331 != 18374858748799134293 2024-09-10T18:29:45.154 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.000+0000 7fd9119d4640 1 -- 172.21.15.115:0/1610273572 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd90c104710 msgr2=0x7fd90c106af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:29:45.154 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.000+0000 7fd9119d4640 1 --2- 172.21.15.115:0/1610273572 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd90c104710 0x7fd90c106af0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc004950 tx=0x7fd8fc009da0 comp rx=0 tx=0).stop 2024-09-10T18:29:45.154 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.000+0000 7fd9119d4640 1 -- 172.21.15.115:0/1610273572 shutdown_connections 2024-09-10T18:29:45.154 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.000+0000 7fd9119d4640 1 --2- 172.21.15.115:0/1610273572 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fd90c109b50 0x7fd90c10c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:29:45.154 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.000+0000 7fd9119d4640 1 --2- 172.21.15.115:0/1610273572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd90c1070c0 0x7fd90c1094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:29:45.154 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.000+0000 7fd9119d4640 1 --2- 172.21.15.115:0/1610273572 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd90c104710 0x7fd90c106af0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:29:45.154 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.000+0000 7fd9119d4640 1 -- 172.21.15.115:0/1610273572 >> 172.21.15.115:0/1610273572 conn(0x7fd90c06a8d0 msgr2=0x7fd90c06cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:29:45.155 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.000+0000 7fd9119d4640 1 -- 172.21.15.115:0/1610273572 shutdown_connections 2024-09-10T18:29:45.155 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.000+0000 7fd9119d4640 1 -- 172.21.15.115:0/1610273572 wait complete. 2024-09-10T18:29:45.155 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.000+0000 7fd9119d4640 1 Processor -- start 2024-09-10T18:29:45.155 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.001+0000 7fd9119d4640 1 -- start start 2024-09-10T18:29:45.155 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.001+0000 7fd9119d4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd90c104710 0x7fd90c10f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:29:45.156 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.001+0000 7fd9119d4640 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fd90c1070c0 0x7fd90c10dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:29:45.156 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.001+0000 7fd9119d4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd90c109b50 0x7fd90c10e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:29:45.156 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.001+0000 7fd9119d4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90c10e860 con 0x7fd90c104710 2024-09-10T18:29:45.156 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.001+0000 7fd9119d4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90c10e9d0 con 0x7fd90c109b50 2024-09-10T18:29:45.156 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.001+0000 7fd9119d4640 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd90c11f730 con 0x7fd90c1070c0 2024-09-10T18:29:45.156 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.001+0000 7fd90affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd90c104710 0x7fd90c10f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:29:45.156 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.001+0000 7fd903fff640 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fd90c1070c0 0x7fd90c10dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:29:45.156 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.001+0000 7fd90affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd90c104710 0x7fd90c10f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41252/0 (socket says 172.21.15.115:41252) 2024-09-10T18:29:45.156 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.001+0000 7fd90affd640 1 -- 172.21.15.115:0/3515761046 learned_addr learned my addr 172.21.15.115:0/3515761046 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-09-10T18:29:45.156 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.001+0000 7fd90b7fe640 1 --2- 172.21.15.115:0/3515761046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd90c109b50 0x7fd90c10e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:29:45.156 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.001+0000 7fd90affd640 1 -- 172.21.15.115:0/3515761046 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fd90c1070c0 msgr2=0x7fd90c10dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:29:45.157 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.001+0000 7fd90affd640 1 --2- 172.21.15.115:0/3515761046 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fd90c1070c0 0x7fd90c10dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:29:45.157 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.001+0000 7fd90affd640 1 -- 172.21.15.115:0/3515761046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd90c109b50 msgr2=0x7fd90c10e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:29:45.157 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.001+0000 7fd90affd640 1 --2- 172.21.15.115:0/3515761046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd90c109b50 0x7fd90c10e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:29:45.157 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.001+0000 7fd90affd640 1 -- 172.21.15.115:0/3515761046 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9040098b0 con 0x7fd90c104710 2024-09-10T18:29:45.157 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.001+0000 7fd903fff640 1 --2- 172.21.15.115:0/3515761046 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fd90c1070c0 0x7fd90c10dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:29:45.157 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.001+0000 7fd90b7fe640 1 --2- 172.21.15.115:0/3515761046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd90c109b50 0x7fd90c10e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:29:45.157 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.001+0000 7fd90affd640 1 --2- 172.21.15.115:0/3515761046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd90c104710 0x7fd90c10f640 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc0043d0 tx=0x7fd8fc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:29:45.157 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.001+0000 7fd9037fe640 1 -- 172.21.15.115:0/3515761046 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc02a960 con 0x7fd90c104710 2024-09-10T18:29:45.157 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.001+0000 7fd9037fe640 1 -- 172.21.15.115:0/3515761046 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- auth(proto 2 2 bytes epoch 0) v1 -- 0x7fd8f8002070 con 0x7fd90c104710 2024-09-10T18:29:45.157 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.001+0000 7fd9119d4640 1 -- 172.21.15.115:0/3515761046 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8fc005040 con 0x7fd90c104710 2024-09-10T18:29:45.157 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.001+0000 7fd9037fe640 1 -- 172.21.15.115:0/3515761046 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(38 keys) v1 ==== 1752+0+0 (secure 0 0 0) 0x7fd8fc009820 con 0x7fd90c104710 2024-09-10T18:29:45.157 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.001+0000 7fd9037fe640 1 -- 172.21.15.115:0/3515761046 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc03c690 con 0x7fd90c104710 2024-09-10T18:29:45.157 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.002+0000 7fd9119d4640 1 -- 172.21.15.115:0/3515761046 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd90c11fbf0 con 0x7fd90c104710 2024-09-10T18:29:45.157 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.002+0000 7fd9037fe640 1 -- 172.21.15.115:0/3515761046 <== mon.0 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x7fd90c11fbf0 con 0x7fd90c104710 2024-09-10T18:29:45.158 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.002+0000 7fd9037fe640 0 cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 1648168043697291331 != 18374858748799134293 2024-09-10T18:29:45.158 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.003+0000 7fd9037fe640 1 -- 172.21.15.115:0/3515761046 <== mon.0 v2:172.21.15.115:3300/0 5 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7fd8fc04c020 con 0x7fd90c104710 2024-09-10T18:29:45.158 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.003+0000 7fd9037fe640 1 --2- 172.21.15.115:0/3515761046 >> v2:172.21.15.115:6800/634497644 conn(0x7fd8f806f120 0x7fd8f80715d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:29:45.158 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.003+0000 7fd9037fe640 1 -- 172.21.15.115:0/3515761046 --> v2:172.21.15.115:6800/634497644 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd8f8071ca0 con 0x7fd8f806f120 2024-09-10T18:29:45.158 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.003+0000 7fd9037fe640 1 -- 172.21.15.115:0/3515761046 <== mon.0 v2:172.21.15.115:3300/0 6 ==== osd_map(63..63 src has 1..63) v4 ==== 6284+0+0 (secure 0 0 0) 0x7fd8fc0af720 con 0x7fd90c104710 2024-09-10T18:29:45.158 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.004+0000 7fd903fff640 1 --2- 172.21.15.115:0/3515761046 >> v2:172.21.15.115:6800/634497644 conn(0x7fd8f806f120 0x7fd8f80715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:29:45.158 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.004+0000 7fd903fff640 1 --2- 172.21.15.115:0/3515761046 >> v2:172.21.15.115:6800/634497644 conn(0x7fd8f806f120 0x7fd8f80715d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4004fd0 tx=0x7fd8f400b040 comp rx=0 tx=0).ready entity=mgr.14538 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:29:45.158 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.004+0000 7fd9037fe640 1 -- 172.21.15.115:0/3515761046 <== mgr.14538 v2:172.21.15.115:6800/634497644 1 ==== command_reply(tid 0: -13 access denied: does your client key have mgr caps? See http://docs.ceph.com/en/latest/mgr/administrator/#client-authentication) v1 ==== 134+0+0 (secure 0 0 0) 0x7fd8f8071ca0 con 0x7fd8f806f120 2024-09-10T18:29:45.158 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: Error EACCES: problem getting command descriptions from mgr.y 2024-09-10T18:29:45.158 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.158 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.018+0000 7faebd667640 -1 mgr handle_mgr_map I was active but no longer am 2024-09-10T18:29:45.158 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ignoring --setuser ceph since I am not root 2024-09-10T18:29:45.158 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: ignoring --setgroup ceph since I am not root 2024-09-10T18:29:45.159 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mgr[29877]: -- 172.21.15.115:0/3882570690 <== mon.0 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b18023e340 con 0x55b18024a800 2024-09-10T18:29:45.159 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mgr[29877]: cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 1648168043697291331 != 18374858748799134293 2024-09-10T18:29:45.159 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.150+0000 7ffa6ed58ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:29:45.305 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:45 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: [10/Sep/2024:18:29:45] ENGINE Bus STOPPING 2024-09-10T18:29:45.305 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:45 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: [10/Sep/2024:18:29:45] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:29:45.305 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:45 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: [10/Sep/2024:18:29:45] ENGINE Bus STOPPED 2024-09-10T18:29:45.305 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:45 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: [10/Sep/2024:18:29:45] ENGINE Bus STARTING 2024-09-10T18:29:45.465 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.208+0000 7ffa6ed58ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:29:45.465 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.274+0000 7ffa6ed58ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:29:45.465 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.397+0000 7ffa6ed58ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:29:45.735 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.536+0000 7ffa6ed58ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:29:45.805 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:45 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: [10/Sep/2024:18:29:45] ENGINE Serving on http://:::9283 2024-09-10T18:29:45.805 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:45 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: [10/Sep/2024:18:29:45] ENGINE Bus STARTED 2024-09-10T18:29:46.185 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.806+0000 7ffa6ed58ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:29:46.185 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:45.872+0000 7ffa6ed58ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:29:46.185 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:46 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:46.043+0000 7ffa6ed58ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:29:46.186 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:46.186 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:46.186 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[29603]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-10T18:29:46.186 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-10T18:29:46.186 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[29603]: Activating manager daemon x 2024-09-10T18:29:46.186 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[29603]: osdmap e64: 8 total, 8 up, 8 in 2024-09-10T18:29:46.186 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[29603]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-09-10T18:29:46.187 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[29603]: mgrmap e22: x(active, starting, since 0.00819692s) 2024-09-10T18:29:46.187 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:29:46.187 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:29:46.187 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:29:46.187 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:29:46.187 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:29:46.187 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:29:46.187 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:29:46.187 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:29:46.187 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:29:46.188 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:29:46.188 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:29:46.188 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:29:46.188 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:29:46.188 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:29:46.188 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:29:46.188 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[29603]: Manager daemon x is now available 2024-09-10T18:29:46.189 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:29:46.189 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:29:46.189 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:29:46.189 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:29:46.190 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:46.190 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:46.190 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[41073]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-10T18:29:46.190 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-10T18:29:46.190 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[41073]: Activating manager daemon x 2024-09-10T18:29:46.191 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[41073]: osdmap e64: 8 total, 8 up, 8 in 2024-09-10T18:29:46.191 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[41073]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-09-10T18:29:46.191 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[41073]: mgrmap e22: x(active, starting, since 0.00819692s) 2024-09-10T18:29:46.191 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:29:46.191 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:29:46.192 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:29:46.192 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:29:46.192 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:29:46.192 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:29:46.192 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:29:46.192 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:29:46.192 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:29:46.192 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:29:46.192 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:29:46.193 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:29:46.193 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:29:46.193 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:29:46.193 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:29:46.193 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[41073]: Manager daemon x is now available 2024-09-10T18:29:46.193 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:29:46.193 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:29:46.193 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:29:46.194 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:29:46.194 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T18:29:46.194 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T18:29:46.194 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T18:29:46.194 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:45 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T18:29:46.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:45 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:46.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:45 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:29:46.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:45 smithi150 ceph-mon[32761]: from='mgr.14538 172.21.15.115:0/1770345206' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-10T18:29:46.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:45 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-10T18:29:46.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:45 smithi150 ceph-mon[32761]: Activating manager daemon x 2024-09-10T18:29:46.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:45 smithi150 ceph-mon[32761]: osdmap e64: 8 total, 8 up, 8 in 2024-09-10T18:29:46.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:45 smithi150 ceph-mon[32761]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-09-10T18:29:46.212 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:45 smithi150 ceph-mon[32761]: mgrmap e22: x(active, starting, since 0.00819692s) 2024-09-10T18:29:46.212 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:45 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:29:46.212 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:45 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:29:46.212 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:45 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:29:46.212 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:45 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:29:46.213 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:45 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:29:46.213 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:45 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:29:46.213 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:45 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:29:46.213 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:45 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:29:46.213 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:45 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:29:46.213 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:45 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:29:46.214 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:45 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:29:46.214 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:45 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:29:46.214 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:45 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:29:46.214 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:45 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:29:46.214 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:45 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:29:46.215 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:45 smithi150 ceph-mon[32761]: Manager daemon x is now available 2024-09-10T18:29:46.215 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:45 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:29:46.215 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:45 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:29:46.216 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:45 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:29:46.217 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:45 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:29:46.217 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:45 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T18:29:46.217 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:45 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T18:29:46.555 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:46 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:46.271+0000 7fc7138be640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:18:29:46] ENGINE Error in HTTPServer.serve 2024-09-10T18:29:46.555 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:46 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: Traceback (most recent call last): 2024-09-10T18:29:46.555 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:46 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:29:46.555 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:46 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: self._connections.run(self.expiration_interval) 2024-09-10T18:29:46.555 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:46 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:29:46.555 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:46 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: self._run(expiration_interval) 2024-09-10T18:29:46.555 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:46 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:29:46.556 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:46 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:29:46.556 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:46 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:29:46.556 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:46 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:29:46.556 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:46 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:29:46.556 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:46 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: s = self.context.wrap_socket( 2024-09-10T18:29:46.556 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:46 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:29:46.556 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:46 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: return self.sslsocket_class._create( 2024-09-10T18:29:46.556 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:46 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:29:46.556 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:46 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: self.do_handshake() 2024-09-10T18:29:46.556 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:46 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:29:46.556 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:46 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: self._sslobj.do_handshake() 2024-09-10T18:29:46.556 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:46 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:29:46.556 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:46 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:47.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:46 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:46.945+0000 7ffa6ed58ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:29:47.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:47 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:47.024+0000 7ffa6ed58ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:29:47.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:47 smithi115 ceph-mon[29603]: mgrmap e23: x(active, since 1.0082s) 2024-09-10T18:29:47.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:47 smithi115 ceph-mon[29603]: [10/Sep/2024:18:29:46] ENGINE Bus STARTING 2024-09-10T18:29:47.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:47 smithi115 ceph-mon[29603]: [10/Sep/2024:18:29:46] ENGINE Serving on https://172.21.15.150:7150 2024-09-10T18:29:47.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:47 smithi115 ceph-mon[29603]: [10/Sep/2024:18:29:46] ENGINE Bus STARTED 2024-09-10T18:29:47.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:47 smithi115 ceph-mon[29603]: [10/Sep/2024:18:29:46] ENGINE Error in HTTPServer.serve 2024-09-10T18:29:47.236 INFO:journalctl@ceph.mon.a.smithi115.stdout: Traceback (most recent call last): 2024-09-10T18:29:47.236 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:29:47.236 INFO:journalctl@ceph.mon.a.smithi115.stdout: self._connections.run(self.expiration_interval) 2024-09-10T18:29:47.236 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:29:47.236 INFO:journalctl@ceph.mon.a.smithi115.stdout: self._run(expiration_interval) 2024-09-10T18:29:47.236 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:29:47.236 INFO:journalctl@ceph.mon.a.smithi115.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:29:47.236 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:29:47.236 INFO:journalctl@ceph.mon.a.smithi115.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:29:47.236 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:29:47.236 INFO:journalctl@ceph.mon.a.smithi115.stdout: s = self.context.wrap_socket( 2024-09-10T18:29:47.236 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:29:47.236 INFO:journalctl@ceph.mon.a.smithi115.stdout: return self.sslsocket_class._create( 2024-09-10T18:29:47.236 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:29:47.237 INFO:journalctl@ceph.mon.a.smithi115.stdout: self.do_handshake() 2024-09-10T18:29:47.237 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:29:47.237 INFO:journalctl@ceph.mon.a.smithi115.stdout: self._sslobj.do_handshake() 2024-09-10T18:29:47.237 INFO:journalctl@ceph.mon.a.smithi115.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:29:47.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:47 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:47.237 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:47 smithi115 ceph-mon[41073]: mgrmap e23: x(active, since 1.0082s) 2024-09-10T18:29:47.237 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:47 smithi115 ceph-mon[41073]: [10/Sep/2024:18:29:46] ENGINE Bus STARTING 2024-09-10T18:29:47.237 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:47 smithi115 ceph-mon[41073]: [10/Sep/2024:18:29:46] ENGINE Serving on https://172.21.15.150:7150 2024-09-10T18:29:47.237 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:47 smithi115 ceph-mon[41073]: [10/Sep/2024:18:29:46] ENGINE Bus STARTED 2024-09-10T18:29:47.237 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:47 smithi115 ceph-mon[41073]: [10/Sep/2024:18:29:46] ENGINE Error in HTTPServer.serve 2024-09-10T18:29:47.237 INFO:journalctl@ceph.mon.c.smithi115.stdout: Traceback (most recent call last): 2024-09-10T18:29:47.238 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:29:47.238 INFO:journalctl@ceph.mon.c.smithi115.stdout: self._connections.run(self.expiration_interval) 2024-09-10T18:29:47.238 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:29:47.238 INFO:journalctl@ceph.mon.c.smithi115.stdout: self._run(expiration_interval) 2024-09-10T18:29:47.238 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:29:47.238 INFO:journalctl@ceph.mon.c.smithi115.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:29:47.238 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:29:47.238 INFO:journalctl@ceph.mon.c.smithi115.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:29:47.238 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:29:47.238 INFO:journalctl@ceph.mon.c.smithi115.stdout: s = self.context.wrap_socket( 2024-09-10T18:29:47.238 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:29:47.238 INFO:journalctl@ceph.mon.c.smithi115.stdout: return self.sslsocket_class._create( 2024-09-10T18:29:47.238 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:29:47.238 INFO:journalctl@ceph.mon.c.smithi115.stdout: self.do_handshake() 2024-09-10T18:29:47.239 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:29:47.239 INFO:journalctl@ceph.mon.c.smithi115.stdout: self._sslobj.do_handshake() 2024-09-10T18:29:47.239 INFO:journalctl@ceph.mon.c.smithi115.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:29:47.239 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:47 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:47.289 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:47 smithi150 ceph-mon[32761]: mgrmap e23: x(active, since 1.0082s) 2024-09-10T18:29:47.289 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:47 smithi150 ceph-mon[32761]: [10/Sep/2024:18:29:46] ENGINE Bus STARTING 2024-09-10T18:29:47.289 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:47 smithi150 ceph-mon[32761]: [10/Sep/2024:18:29:46] ENGINE Serving on https://172.21.15.150:7150 2024-09-10T18:29:47.289 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:47 smithi150 ceph-mon[32761]: [10/Sep/2024:18:29:46] ENGINE Bus STARTED 2024-09-10T18:29:47.290 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:47 smithi150 ceph-mon[32761]: [10/Sep/2024:18:29:46] ENGINE Error in HTTPServer.serve 2024-09-10T18:29:47.290 INFO:journalctl@ceph.mon.b.smithi150.stdout: Traceback (most recent call last): 2024-09-10T18:29:47.290 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:29:47.290 INFO:journalctl@ceph.mon.b.smithi150.stdout: self._connections.run(self.expiration_interval) 2024-09-10T18:29:47.290 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:29:47.290 INFO:journalctl@ceph.mon.b.smithi150.stdout: self._run(expiration_interval) 2024-09-10T18:29:47.290 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:29:47.290 INFO:journalctl@ceph.mon.b.smithi150.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:29:47.290 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:29:47.290 INFO:journalctl@ceph.mon.b.smithi150.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:29:47.290 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:29:47.290 INFO:journalctl@ceph.mon.b.smithi150.stdout: s = self.context.wrap_socket( 2024-09-10T18:29:47.290 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:29:47.290 INFO:journalctl@ceph.mon.b.smithi150.stdout: return self.sslsocket_class._create( 2024-09-10T18:29:47.291 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:29:47.291 INFO:journalctl@ceph.mon.b.smithi150.stdout: self.do_handshake() 2024-09-10T18:29:47.291 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:29:47.291 INFO:journalctl@ceph.mon.b.smithi150.stdout: self._sslobj.do_handshake() 2024-09-10T18:29:47.291 INFO:journalctl@ceph.mon.b.smithi150.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:29:47.291 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:47 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:48.057 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:47 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:47.750+0000 7ffa6ed58ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:29:48.057 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:47 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:47.871+0000 7ffa6ed58ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:29:48.057 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:47 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:47.992+0000 7ffa6ed58ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:29:48.058 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:48 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:48.055+0000 7ffa6ed58ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:29:48.262 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key mgr.y 2024-09-10T18:29:48.333 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:48 smithi115 ceph-mon[41073]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:29:48.333 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:48 smithi115 ceph-mon[41073]: mgrmap e24: x(active, since 2s) 2024-09-10T18:29:48.333 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:48 smithi115 ceph-mon[29603]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:29:48.333 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:48 smithi115 ceph-mon[29603]: mgrmap e24: x(active, since 2s) 2024-09-10T18:29:48.334 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:48 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:48.188+0000 7ffa6ed58ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:29:48.334 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:48 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:48.259+0000 7ffa6ed58ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:29:48.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:48 smithi150 ceph-mon[32761]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:29:48.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:48 smithi150 ceph-mon[32761]: mgrmap e24: x(active, since 2s) 2024-09-10T18:29:48.622 INFO:teuthology.orchestra.run.smithi115.stdout:rotating key for mgr.x 2024-09-10T18:29:48.623 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAXkOBm4tS1ExAAcigcZopKYK7T3BnY6IQZeQ== 2024-09-10T18:29:48.623 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQD2jOBmmh07NBAAr+VG9WQKWtbLqgijPlotBw== == AQAXkOBm4tS1ExAAcigcZopKYK7T3BnY6IQZeQ== ']' 2024-09-10T18:29:48.623 INFO:teuthology.orchestra.run.smithi115.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-10T18:29:48.623 INFO:teuthology.orchestra.run.smithi115.stderr:+ echo 'rotating key for mgr.x' 2024-09-10T18:29:48.623 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:48 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:48.331+0000 7ffa6ed58ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:29:48.623 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:48 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:48.502+0000 7ffa6ed58ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:29:48.623 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:48 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:48.569+0000 7ffa6ed58ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:29:48.624 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key mgr.x 2024-09-10T18:29:48.949 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:48 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:48.642+0000 7ffa6ed58ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:29:48.949 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:48 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:48.730+0000 7ffa6ed58ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:29:48.949 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:48 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:48.792+0000 7ffa6ed58ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:29:48.949 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:48 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:48.855+0000 7ffa6ed58ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:29:48.949 INFO:teuthology.orchestra.run.smithi115.stderr:+ K=AQBpjeBmS0Q8JBAA43UXuB8ps4Yo5rHYcmJapA== 2024-09-10T18:29:48.950 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBpjeBmS0Q8JBAA43UXuB8ps4Yo5rHYcmJapA== 2024-09-10T18:29:48.950 INFO:teuthology.orchestra.run.smithi115.stderr:+ ceph orch daemon rotate-key mgr.x 2024-09-10T18:29:49.054 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:48 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:29:48] "GET /metrics HTTP/1.1" 200 34931 "" "Prometheus/2.43.0" 2024-09-10T18:29:49.220 INFO:teuthology.orchestra.run.smithi115.stdout:Scheduled to rotate-key mgr.x on host 'smithi150' 2024-09-10T18:29:49.232 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBpjeBmS0Q8JBAA43UXuB8ps4Yo5rHYcmJapA== == AQBpjeBmS0Q8JBAA43UXuB8ps4Yo5rHYcmJapA== ']' 2024-09-10T18:29:49.233 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:29:49.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:49 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:49.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:49 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:49.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:49 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/2606466674' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-10T18:29:49.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:49 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/1263167304' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-10T18:29:49.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:49 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:49.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:49 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:49.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:49 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:49.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:49 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:49.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:49 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:49.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:49 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:49.807 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:49 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:49.807 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:49 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:49.807 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:49 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:49.807 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:49 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:49.807 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:49 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:49.807 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:49 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:49.807 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:49 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:49.985 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:49.645+0000 7ffa6ed58ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:29:49.985 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: [10/Sep/2024:18:29:49] ENGINE Bus STARTING 2024-09-10T18:29:49.985 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: CherryPy Checker: 2024-09-10T18:29:49.985 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: The Application mounted at '' has an empty config. 2024-09-10T18:29:49.985 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: 2024-09-10T18:29:49.985 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: [10/Sep/2024:18:29:49] ENGINE Serving on http://:::9283 2024-09-10T18:29:49.985 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[29854]: [10/Sep/2024:18:29:49] ENGINE Bus STARTED 2024-09-10T18:29:49.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:49.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:49.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/2606466674' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-10T18:29:49.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/1263167304' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-10T18:29:49.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:49.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:49.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:49.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:49.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:49.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:49.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:49.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:49.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:49.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:49.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:49.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:49.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:49.987 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:49.987 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:49.987 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/2606466674' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-10T18:29:49.987 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/1263167304' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-10T18:29:49.987 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:49.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:49.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:49.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:49.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:49.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:49.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:49.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:49.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:49.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:49.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:49.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:49.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:49 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:50.804 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:29:50 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:29:50.810 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:50 smithi150 ceph-mon[32761]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:29:50.810 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:50 smithi150 ceph-mon[32761]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:29:50.810 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:50 smithi150 ceph-mon[32761]: Schedule rotate-key daemon mgr.x 2024-09-10T18:29:50.810 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:50 smithi150 ceph-mon[32761]: Adjusting osd_memory_target on smithi115 to 3248M 2024-09-10T18:29:50.810 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:50 smithi150 ceph-mon[32761]: mgrmap e25: x(active, since 4s) 2024-09-10T18:29:50.810 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:50 smithi150 ceph-mon[32761]: Standby manager daemon y started 2024-09-10T18:29:50.810 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:50 smithi150 ceph-mon[32761]: from='mgr.? 172.21.15.115:0/2757258910' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T18:29:50.810 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:50 smithi150 ceph-mon[32761]: from='mgr.? 172.21.15.115:0/2757258910' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:29:50.810 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:50 smithi150 ceph-mon[32761]: from='mgr.? 172.21.15.115:0/2757258910' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T18:29:50.811 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:50 smithi150 ceph-mon[32761]: from='mgr.? 172.21.15.115:0/2757258910' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:29:50.811 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:50 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:50.811 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:50 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:50.811 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:50 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:50.811 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:50 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:50.811 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:50 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:50.811 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:50 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:50.811 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:50 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:50.811 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:50 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:50.811 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:50 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:50.811 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:50 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:50.811 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:50 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:50.811 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:50 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:50.811 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:50 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:50.812 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:50 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:29:50.812 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:50 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:29:50.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[41073]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:29:50.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[41073]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:29:50.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[41073]: Schedule rotate-key daemon mgr.x 2024-09-10T18:29:50.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[41073]: Adjusting osd_memory_target on smithi115 to 3248M 2024-09-10T18:29:50.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[41073]: mgrmap e25: x(active, since 4s) 2024-09-10T18:29:50.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[41073]: Standby manager daemon y started 2024-09-10T18:29:50.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[41073]: from='mgr.? 172.21.15.115:0/2757258910' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T18:29:50.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[41073]: from='mgr.? 172.21.15.115:0/2757258910' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:29:50.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[41073]: from='mgr.? 172.21.15.115:0/2757258910' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T18:29:50.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[41073]: from='mgr.? 172.21.15.115:0/2757258910' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:29:50.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:50.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:50.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:50.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:50.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:50.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:50.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:50.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:50.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:50.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:50.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:50.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:50.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:50.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:29:50.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:29:50.987 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[29603]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:29:50.987 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[29603]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:29:50.987 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[29603]: Schedule rotate-key daemon mgr.x 2024-09-10T18:29:50.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[29603]: Adjusting osd_memory_target on smithi115 to 3248M 2024-09-10T18:29:50.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[29603]: mgrmap e25: x(active, since 4s) 2024-09-10T18:29:50.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[29603]: Standby manager daemon y started 2024-09-10T18:29:50.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[29603]: from='mgr.? 172.21.15.115:0/2757258910' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T18:29:50.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[29603]: from='mgr.? 172.21.15.115:0/2757258910' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:29:50.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[29603]: from='mgr.? 172.21.15.115:0/2757258910' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T18:29:50.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[29603]: from='mgr.? 172.21.15.115:0/2757258910' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:29:50.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:50.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:50.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:50.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:50.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:50.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:50.993 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:50.993 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:50.993 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:50.993 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:50.993 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:50.993 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:50.993 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:50.993 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:29:50.994 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:50 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:29:51.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:51 smithi115 ceph-mon[41073]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:29:51.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:51 smithi115 ceph-mon[41073]: Adjusting osd_memory_target on smithi150 to 3248M 2024-09-10T18:29:51.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:51 smithi115 ceph-mon[41073]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-10T18:29:51.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:51 smithi115 ceph-mon[41073]: Updating smithi150:/etc/ceph/ceph.conf 2024-09-10T18:29:51.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:51 smithi115 ceph-mon[41073]: mgrmap e26: x(active, since 5s), standbys: y 2024-09-10T18:29:51.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:51 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:29:51.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:51 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:51.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:51 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:51.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:51 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:51.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:51 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:51.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:51 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:51.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:51 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-10T18:29:51.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:51 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-10T18:29:51.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:51 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-10T18:29:51.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:51 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:29:51.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:51 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:29:51.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:51 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:29:51.986 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:51 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:29:51.986 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:51 smithi115 ceph-mon[29603]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:29:51.987 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:51 smithi115 ceph-mon[29603]: Adjusting osd_memory_target on smithi150 to 3248M 2024-09-10T18:29:51.987 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:51 smithi115 ceph-mon[29603]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-10T18:29:51.987 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:51 smithi115 ceph-mon[29603]: Updating smithi150:/etc/ceph/ceph.conf 2024-09-10T18:29:51.987 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:51 smithi115 ceph-mon[29603]: mgrmap e26: x(active, since 5s), standbys: y 2024-09-10T18:29:51.987 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:51 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:29:51.987 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:51 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:51.987 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:51 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:51.987 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:51 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:51.987 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:51 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:51.987 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:51 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:51.987 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:51 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-10T18:29:51.987 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:51 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-10T18:29:51.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:51 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-10T18:29:51.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:51 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:29:51.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:51 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:29:51.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:51 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:29:51.988 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:51 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:29:52.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:51 smithi150 ceph-mon[32761]: from='client.24410 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:29:52.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:51 smithi150 ceph-mon[32761]: Adjusting osd_memory_target on smithi150 to 3248M 2024-09-10T18:29:52.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:51 smithi150 ceph-mon[32761]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-10T18:29:52.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:51 smithi150 ceph-mon[32761]: Updating smithi150:/etc/ceph/ceph.conf 2024-09-10T18:29:52.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:51 smithi150 ceph-mon[32761]: mgrmap e26: x(active, since 5s), standbys: y 2024-09-10T18:29:52.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:51 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:29:52.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:51 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:52.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:51 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:52.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:51 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:52.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:51 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:52.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:51 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:52.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:51 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-10T18:29:52.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:51 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-10T18:29:52.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:51 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-10T18:29:52.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:51 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:29:52.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:51 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:29:52.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:51 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:29:52.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:51 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:29:52.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:52 smithi115 ceph-mon[41073]: Updating smithi150:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:29:52.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:52 smithi115 ceph-mon[41073]: Updating smithi115:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:29:52.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:52 smithi115 ceph-mon[41073]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:29:52.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:52 smithi115 ceph-mon[41073]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:29:52.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:52 smithi115 ceph-mon[41073]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T18:29:52.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:52 smithi115 ceph-mon[41073]: Updating smithi150:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:29:52.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:52 smithi115 ceph-mon[41073]: Updating smithi115:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:29:52.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:52 smithi115 ceph-mon[41073]: Rotating authentication key for mgr.y 2024-09-10T18:29:52.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:52 smithi115 ceph-mon[41073]: Reconfiguring daemon mgr.y on smithi115 2024-09-10T18:29:52.939 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:52 smithi115 ceph-mon[29603]: Updating smithi150:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:29:52.939 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:52 smithi115 ceph-mon[29603]: Updating smithi115:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:29:52.939 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:52 smithi115 ceph-mon[29603]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:29:52.939 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:52 smithi115 ceph-mon[29603]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:29:52.940 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:52 smithi115 ceph-mon[29603]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T18:29:52.940 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:52 smithi115 ceph-mon[29603]: Updating smithi150:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:29:52.940 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:52 smithi115 ceph-mon[29603]: Updating smithi115:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:29:52.940 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:52 smithi115 ceph-mon[29603]: Rotating authentication key for mgr.y 2024-09-10T18:29:52.940 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:52 smithi115 ceph-mon[29603]: Reconfiguring daemon mgr.y on smithi115 2024-09-10T18:29:53.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:52 smithi150 ceph-mon[32761]: Updating smithi150:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:29:53.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:52 smithi150 ceph-mon[32761]: Updating smithi115:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:29:53.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:52 smithi150 ceph-mon[32761]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:29:53.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:52 smithi150 ceph-mon[32761]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:29:53.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:52 smithi150 ceph-mon[32761]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T18:29:53.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:52 smithi150 ceph-mon[32761]: Updating smithi150:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:29:53.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:52 smithi150 ceph-mon[32761]: Updating smithi115:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:29:53.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:52 smithi150 ceph-mon[32761]: Rotating authentication key for mgr.y 2024-09-10T18:29:53.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:52 smithi150 ceph-mon[32761]: Reconfiguring daemon mgr.y on smithi115 2024-09-10T18:29:53.890 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:53 smithi115 systemd[1]: Stopping Ceph mgr.y for a600e180-6fa0-11ef-bcea-c7b262605968... 2024-09-10T18:29:54.150 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:54 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:54.151 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:54 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:54.151 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:54 smithi115 ceph-mon[41073]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T18:29:54.151 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:53 smithi115 podman[86348]: 2024-09-10 18:29:53.888657274 +0000 UTC m=+0.120738746 container stop bcf84116aaa96ed215eb03dc40d579f6c756560a6abb2f16801f553a469b4598 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:29:54.151 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:53 smithi115 podman[86348]: 2024-09-10 18:29:53.915124703 +0000 UTC m=+0.147206177 container died bcf84116aaa96ed215eb03dc40d579f6c756560a6abb2f16801f553a469b4598 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-10T18:29:54.151 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:54 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:54.151 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:54 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:54.151 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:54 smithi115 ceph-mon[29603]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T18:29:54.235 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key mgr.x 2024-09-10T18:29:54.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:54 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:54.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:54 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:54.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:54 smithi150 ceph-mon[32761]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T18:29:54.427 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:54 smithi115 podman[86348]: 2024-09-10 18:29:54.239945721 +0000 UTC m=+0.472027193 container cleanup bcf84116aaa96ed215eb03dc40d579f6c756560a6abb2f16801f553a469b4598 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T18:29:54.427 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:54 smithi115 bash[86348]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y 2024-09-10T18:29:54.549 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBpjeBmS0Q8JBAA43UXuB8ps4Yo5rHYcmJapA== 2024-09-10T18:29:54.549 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBpjeBmS0Q8JBAA43UXuB8ps4Yo5rHYcmJapA== == AQBpjeBmS0Q8JBAA43UXuB8ps4Yo5rHYcmJapA== ']' 2024-09-10T18:29:54.549 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T18:29:54.735 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:54 smithi115 podman[86367]: 2024-09-10 18:29:54.425555828 +0000 UTC m=+0.530958168 container remove bcf84116aaa96ed215eb03dc40d579f6c756560a6abb2f16801f553a469b4598 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-10T18:29:54.735 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:54 smithi115 systemd[1]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-10T18:29:54.735 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:54 smithi115 systemd[1]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-09-10T18:29:54.735 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:54 smithi115 systemd[1]: Stopped Ceph mgr.y for a600e180-6fa0-11ef-bcea-c7b262605968. 2024-09-10T18:29:54.735 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:54 smithi115 systemd[1]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968@mgr.y.service: Consumed 43.070s CPU time. 2024-09-10T18:29:54.735 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:54 smithi115 systemd[1]: Starting Ceph mgr.y for a600e180-6fa0-11ef-bcea-c7b262605968... 2024-09-10T18:29:55.138 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:55 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/3673630806' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-10T18:29:55.139 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:55 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/3673630806' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-10T18:29:55.139 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:54 smithi115 podman[86488]: 2024-09-10 18:29:54.768868044 +0000 UTC m=+0.015906622 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:29:55.139 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:54 smithi115 podman[86488]: 2024-09-10 18:29:54.884875004 +0000 UTC m=+0.131913568 container create ba5505bf37f5320044656588855075cdc312c9cfd122d70105d78ec44b0af3b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-10T18:29:55.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:55 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/3673630806' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-10T18:29:55.392 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:55 smithi115 podman[86488]: 2024-09-10 18:29:55.137394391 +0000 UTC m=+0.384432955 container init ba5505bf37f5320044656588855075cdc312c9cfd122d70105d78ec44b0af3b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T18:29:55.392 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:55 smithi115 podman[86488]: 2024-09-10 18:29:55.142180968 +0000 UTC m=+0.389219533 container start ba5505bf37f5320044656588855075cdc312c9cfd122d70105d78ec44b0af3b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-10T18:29:55.392 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:55 smithi115 ceph-mgr[86525]: -- 172.21.15.115:0/991400253 <== mon.0 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5590a8ca6340 con 0x5590a8cb2800 2024-09-10T18:29:55.392 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:55 smithi115 ceph-mgr[86525]: cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 2006320850530184536 != 18374858748799134293 2024-09-10T18:29:55.392 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:55 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: 2024-09-10T18:29:55.254+0000 7fcec85ddec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:29:55.392 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:55 smithi115 bash[86488]: ba5505bf37f5320044656588855075cdc312c9cfd122d70105d78ec44b0af3b2 2024-09-10T18:29:55.392 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:55 smithi115 systemd[1]: Started Ceph mgr.y for a600e180-6fa0-11ef-bcea-c7b262605968. 2024-09-10T18:29:55.392 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:55 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: 2024-09-10T18:29:55.325+0000 7fcec85ddec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:29:55.392 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:55 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: 2024-09-10T18:29:55.389+0000 7fcec85ddec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:29:55.649 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:55 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: 2024-09-10T18:29:55.513+0000 7fcec85ddec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:29:55.907 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:55 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: 2024-09-10T18:29:55.647+0000 7fcec85ddec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:29:56.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:55 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: 2024-09-10T18:29:55.905+0000 7fcec85ddec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:29:56.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:55 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: 2024-09-10T18:29:55.965+0000 7fcec85ddec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:29:56.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:56 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: 2024-09-10T18:29:56.129+0000 7fcec85ddec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:29:56.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-mon[32761]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T18:29:56.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:56.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:56.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-mon[32761]: Rotating authentication key for mgr.x 2024-09-10T18:29:56.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-10T18:29:56.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-10T18:29:56.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]': finished 2024-09-10T18:29:56.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:29:56.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:29:56.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:29:56.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:29:56.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-mon[32761]: Reconfiguring daemon mgr.x on smithi150 2024-09-10T18:29:56.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:56 smithi115 ceph-mon[29603]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T18:29:56.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:56 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:56.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:56 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:56.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:56 smithi115 ceph-mon[29603]: Rotating authentication key for mgr.x 2024-09-10T18:29:56.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:56 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-10T18:29:56.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:56 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-10T18:29:56.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:56 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]': finished 2024-09-10T18:29:56.736 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:56 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:29:56.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:56 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:29:56.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:56 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:29:56.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:56 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:29:56.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:56 smithi115 ceph-mon[29603]: Reconfiguring daemon mgr.x on smithi150 2024-09-10T18:29:56.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:56 smithi115 ceph-mon[41073]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T18:29:56.738 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:56 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:56.738 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:56 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:56.738 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:56 smithi115 ceph-mon[41073]: Rotating authentication key for mgr.x 2024-09-10T18:29:56.738 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:56 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-10T18:29:56.738 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:56 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-10T18:29:56.738 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:56 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]': finished 2024-09-10T18:29:56.739 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:56 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:29:56.739 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:56 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:29:56.739 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:56 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:29:56.739 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:56 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:29:56.739 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:56 smithi115 ceph-mon[41073]: Reconfiguring daemon mgr.x on smithi150 2024-09-10T18:29:56.984 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.834+0000 7fc749129640 -1 mgr.server reply reply (13) Permission denied access denied: does your client key have mgr caps? See http://docs.ceph.com/en/latest/mgr/administrator/#client-authentication 2024-09-10T18:29:56.984 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.836+0000 7fc746924640 -1 log_channel(cephadm) log [ERR] : Non-zero return from ['ceph', '-k', '/var/lib/ceph/mgr/ceph-x/keyring', '-n', 'mgr.x', 'tell', 'mgr.x', 'rotate-key', '-i', '-']: 2024-09-10T18:29:56.830+0000 7f238effd640 1 -- 172.21.15.150:0/3707297273 <== mon.1 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x7f2378003710 con 0x7f2390104710 2024-09-10T18:29:56.984 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.830+0000 7f238effd640 0 cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 16882291417355706972 != 18374858748799134293 2024-09-10T18:29:56.984 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.830+0000 7f23966eb640 1 -- 172.21.15.150:0/3707297273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2390104710 msgr2=0x7f2390106af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:29:56.984 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.830+0000 7f23966eb640 1 --2- 172.21.15.150:0/3707297273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2390104710 0x7f2390106af0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2380004950 tx=0x7f2380009da0 comp rx=0 tx=0).stop 2024-09-10T18:29:56.985 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.830+0000 7f23966eb640 1 -- 172.21.15.150:0/3707297273 shutdown_connections 2024-09-10T18:29:56.985 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.830+0000 7f23966eb640 1 --2- 172.21.15.150:0/3707297273 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f2390109b70 0x7f239010c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:29:56.985 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.830+0000 7f23966eb640 1 --2- 172.21.15.150:0/3707297273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23901070c0 0x7f23901094e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:29:56.985 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.830+0000 7f23966eb640 1 --2- 172.21.15.150:0/3707297273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2390104710 0x7f2390106af0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:29:56.985 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.830+0000 7f23966eb640 1 -- 172.21.15.150:0/3707297273 >> 172.21.15.150:0/3707297273 conn(0x7f239006a8f0 msgr2=0x7f239006cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:29:56.985 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.830+0000 7f23966eb640 1 -- 172.21.15.150:0/3707297273 shutdown_connections 2024-09-10T18:29:56.985 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.830+0000 7f23966eb640 1 -- 172.21.15.150:0/3707297273 wait complete. 2024-09-10T18:29:56.985 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.831+0000 7f23966eb640 1 Processor -- start 2024-09-10T18:29:56.985 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.831+0000 7f23966eb640 1 -- start start 2024-09-10T18:29:56.985 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.831+0000 7f23966eb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2390104710 0x7f23901177a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:29:56.985 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.831+0000 7f23966eb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23901070c0 0x7f2390117ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:29:56.985 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.831+0000 7f23966eb640 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f2390109b70 0x7f239011edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:29:56.986 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.831+0000 7f23966eb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239011f300 con 0x7f2390104710 2024-09-10T18:29:56.986 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.831+0000 7f23966eb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239011f470 con 0x7f23901070c0 2024-09-10T18:29:56.986 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.831+0000 7f23966eb640 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f239011f630 con 0x7f2390109b70 2024-09-10T18:29:56.986 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.831+0000 7f238f7fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23901070c0 0x7f2390117ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:29:56.986 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.831+0000 7f238f7fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23901070c0 0x7f2390117ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53294/0 (socket says 172.21.15.150:53294) 2024-09-10T18:29:56.986 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.831+0000 7f238f7fe640 1 -- 172.21.15.150:0/367471165 learned_addr learned my addr 172.21.15.150:0/367471165 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-09-10T18:29:56.986 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.831+0000 7f238ffff640 1 --2- 172.21.15.150:0/367471165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2390104710 0x7f23901177a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:29:56.986 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.831+0000 7f2394c88640 1 --2- 172.21.15.150:0/367471165 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f2390109b70 0x7f239011edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:29:56.986 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.831+0000 7f238f7fe640 1 -- 172.21.15.150:0/367471165 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f2390109b70 msgr2=0x7f239011edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:29:56.986 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.831+0000 7f238f7fe640 1 --2- 172.21.15.150:0/367471165 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f2390109b70 0x7f239011edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:29:56.987 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.831+0000 7f238f7fe640 1 -- 172.21.15.150:0/367471165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2390104710 msgr2=0x7f23901177a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:29:56.987 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.831+0000 7f238f7fe640 1 --2- 172.21.15.150:0/367471165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2390104710 0x7f23901177a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:29:56.987 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.831+0000 7f238f7fe640 1 -- 172.21.15.150:0/367471165 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2380005040 con 0x7f23901070c0 2024-09-10T18:29:56.987 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.831+0000 7f238ffff640 1 --2- 172.21.15.150:0/367471165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2390104710 0x7f23901177a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:29:56.987 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.831+0000 7f2394c88640 1 --2- 172.21.15.150:0/367471165 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f2390109b70 0x7f239011edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:29:56.987 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.831+0000 7f238f7fe640 1 --2- 172.21.15.150:0/367471165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23901070c0 0x7f2390117ce0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f238400ebf0 tx=0x7f238400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:29:56.987 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.831+0000 7f238cff9640 1 -- 172.21.15.150:0/367471165 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2384007c30 con 0x7f23901070c0 2024-09-10T18:29:56.987 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.832+0000 7f238cff9640 1 -- 172.21.15.150:0/367471165 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- auth(proto 2 2 bytes epoch 0) v1 -- 0x7f23780032d0 con 0x7f23901070c0 2024-09-10T18:29:56.987 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.832+0000 7f238cff9640 1 -- 172.21.15.150:0/367471165 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(38 keys) v1 ==== 1752+0+0 (secure 0 0 0) 0x7f238400f040 con 0x7f23901070c0 2024-09-10T18:29:56.987 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.832+0000 7f23966eb640 1 -- 172.21.15.150:0/367471165 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f239011f8b0 con 0x7f23901070c0 2024-09-10T18:29:56.987 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.832+0000 7f238cff9640 1 -- 172.21.15.150:0/367471165 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23840149c0 con 0x7f23901070c0 2024-09-10T18:29:56.988 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.832+0000 7f23966eb640 1 -- 172.21.15.150:0/367471165 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f239011fdd0 con 0x7f23901070c0 2024-09-10T18:29:56.988 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.832+0000 7f238cff9640 1 -- 172.21.15.150:0/367471165 <== mon.1 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x7f239011fdd0 con 0x7f23901070c0 2024-09-10T18:29:56.988 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.832+0000 7f238cff9640 0 cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 16882291417355706972 != 18374858748799134293 2024-09-10T18:29:56.988 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.833+0000 7f238cff9640 1 -- 172.21.15.150:0/367471165 <== mon.1 v2:172.21.15.150:3300/0 5 ==== mgrmap(e 26) v1 ==== 90804+0+0 (secure 0 0 0) 0x7f238400c770 con 0x7f23901070c0 2024-09-10T18:29:56.988 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.833+0000 7f238cff9640 1 --2- 172.21.15.150:0/367471165 >> v2:172.21.15.150:6816/4142942987 conn(0x7f237806f1c0 0x7f2378071670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:29:56.989 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.833+0000 7f238cff9640 1 -- 172.21.15.150:0/367471165 --> v2:172.21.15.150:6816/4142942987 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2378071d40 con 0x7f237806f1c0 2024-09-10T18:29:56.989 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.833+0000 7f238cff9640 1 -- 172.21.15.150:0/367471165 <== mon.1 v2:172.21.15.150:3300/0 6 ==== osd_map(64..64 src has 1..64) v4 ==== 6499+0+0 (secure 0 0 0) 0x7f238408f210 con 0x7f23901070c0 2024-09-10T18:29:56.989 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.833+0000 7f238ffff640 1 --2- 172.21.15.150:0/367471165 >> v2:172.21.15.150:6816/4142942987 conn(0x7f237806f1c0 0x7f2378071670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:29:56.989 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.834+0000 7f238ffff640 1 --2- 172.21.15.150:0/367471165 >> v2:172.21.15.150:6816/4142942987 conn(0x7f237806f1c0 0x7f2378071670 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f238002a4a0 tx=0x7f2380009da0 comp rx=0 tx=0).ready entity=mgr.24439 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:29:56.989 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.834+0000 7f238cff9640 1 -- 172.21.15.150:0/367471165 <== mgr.24439 v2:172.21.15.150:6816/4142942987 1 ==== command_reply(tid 0: -13 access denied: does your client key have mgr caps? See http://docs.ceph.com/en/latest/mgr/administrator/#client-authentication) v1 ==== 134+0+0 (secure 0 0 0) 0x7f2378071d40 con 0x7f237806f1c0 2024-09-10T18:29:56.989 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: Error EACCES: problem getting command descriptions from mgr.x 2024-09-10T18:29:56.989 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.989 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.847+0000 7fc76bd50640 -1 mgr handle_mgr_map I was active but no longer am 2024-09-10T18:29:56.989 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ignoring --setuser ceph since I am not root 2024-09-10T18:29:56.989 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: ignoring --setgroup ceph since I am not root 2024-09-10T18:29:56.990 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-mgr[34838]: -- 172.21.15.150:0/3193012834 <== mon.1 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562441560340 con 0x56244156c800 2024-09-10T18:29:56.990 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-mgr[34838]: cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 16882291417355706972 != 18374858748799134293 2024-09-10T18:29:56.990 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:56.981+0000 7f9bad94aec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:29:57.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:56 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: 2024-09-10T18:29:56.996+0000 7fcec85ddec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:29:57.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:57 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: 2024-09-10T18:29:57.060+0000 7fcec85ddec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:29:57.305 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:57 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:57.036+0000 7f9bad94aec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:29:57.305 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:57 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:57.103+0000 7f9bad94aec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:29:57.305 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:57 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:57.226+0000 7f9bad94aec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:29:57.618 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:57 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:57.362+0000 7f9bad94aec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:29:58.012 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:57 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: 2024-09-10T18:29:57.780+0000 7fcec85ddec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:29:58.012 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:57 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: 2024-09-10T18:29:57.892+0000 7fcec85ddec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:29:58.013 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:57 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:58.013 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:57 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:58.013 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:57 smithi115 ceph-mon[41073]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-10T18:29:58.013 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:57 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-10T18:29:58.013 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:57 smithi115 ceph-mon[41073]: Activating manager daemon y 2024-09-10T18:29:58.013 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:57 smithi115 ceph-mon[41073]: osdmap e65: 8 total, 8 up, 8 in 2024-09-10T18:29:58.013 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:57 smithi115 ceph-mon[41073]: from='mgr.24439 ' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2024-09-10T18:29:58.013 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:57 smithi115 ceph-mon[41073]: mgrmap e27: y(active, starting, since 0.00911473s) 2024-09-10T18:29:58.013 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:57 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:58.014 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:57 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:58.014 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:57 smithi115 ceph-mon[29603]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-10T18:29:58.014 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:57 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-10T18:29:58.014 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:57 smithi115 ceph-mon[29603]: Activating manager daemon y 2024-09-10T18:29:58.014 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:57 smithi115 ceph-mon[29603]: osdmap e65: 8 total, 8 up, 8 in 2024-09-10T18:29:58.014 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:57 smithi115 ceph-mon[29603]: from='mgr.24439 ' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2024-09-10T18:29:58.014 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:57 smithi115 ceph-mon[29603]: mgrmap e27: y(active, starting, since 0.00911473s) 2024-09-10T18:29:58.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:57 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:57.615+0000 7f9bad94aec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:29:58.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:57 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:57.675+0000 7f9bad94aec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:29:58.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:57 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:57.839+0000 7f9bad94aec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:29:58.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:57 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:58.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:57 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' 2024-09-10T18:29:58.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:57 smithi150 ceph-mon[32761]: from='mgr.24439 172.21.15.150:0/1370003881' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-10T18:29:58.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:57 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-10T18:29:58.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:57 smithi150 ceph-mon[32761]: Activating manager daemon y 2024-09-10T18:29:58.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:57 smithi150 ceph-mon[32761]: osdmap e65: 8 total, 8 up, 8 in 2024-09-10T18:29:58.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:57 smithi150 ceph-mon[32761]: from='mgr.24439 ' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2024-09-10T18:29:58.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:57 smithi150 ceph-mon[32761]: mgrmap e27: y(active, starting, since 0.00911473s) 2024-09-10T18:29:58.269 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:58 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: 2024-09-10T18:29:58.010+0000 7fcec85ddec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:29:58.269 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:58 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: 2024-09-10T18:29:58.071+0000 7fcec85ddec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:29:58.269 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:58 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: 2024-09-10T18:29:58.198+0000 7fcec85ddec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:29:58.525 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:58 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: 2024-09-10T18:29:58.267+0000 7fcec85ddec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:29:58.526 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:58 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: 2024-09-10T18:29:58.326+0000 7fcec85ddec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:29:58.526 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:58 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: 2024-09-10T18:29:58.461+0000 7fcec85ddec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:29:58.782 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:58 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: 2024-09-10T18:29:58.523+0000 7fcec85ddec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:29:58.782 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:58 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: 2024-09-10T18:29:58.588+0000 7fcec85ddec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:29:58.783 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:58 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: 2024-09-10T18:29:58.668+0000 7fcec85ddec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:29:58.783 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:58 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: 2024-09-10T18:29:58.724+0000 7fcec85ddec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:29:59.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:58 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:58.695+0000 7f9bad94aec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:29:59.055 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:58 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:58.759+0000 7f9bad94aec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:29:59.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:58 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: 2024-09-10T18:29:58.780+0000 7fcec85ddec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:29:59.551 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key mgr.x 2024-09-10T18:29:59.735 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: 2024-09-10T18:29:59.540+0000 7fcec85ddec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:29:59.735 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: [10/Sep/2024:18:29:59] ENGINE Bus STARTING 2024-09-10T18:29:59.735 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: CherryPy Checker: 2024-09-10T18:29:59.735 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: The Application mounted at '' has an empty config. 2024-09-10T18:29:59.735 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: 2024-09-10T18:29:59.736 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: [10/Sep/2024:18:29:59] ENGINE Serving on http://:::9283 2024-09-10T18:29:59.736 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: [10/Sep/2024:18:29:59] ENGINE Bus STARTED 2024-09-10T18:29:59.736 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: [10/Sep/2024:18:29:59] ENGINE Bus STOPPING 2024-09-10T18:29:59.736 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: [10/Sep/2024:18:29:59] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:29:59.736 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: [10/Sep/2024:18:29:59] ENGINE Bus STOPPED 2024-09-10T18:29:59.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[41073]: Active manager daemon y restarted 2024-09-10T18:29:59.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[41073]: Activating manager daemon y 2024-09-10T18:29:59.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[41073]: osdmap e66: 8 total, 8 up, 8 in 2024-09-10T18:29:59.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[41073]: mgrmap e28: y(active, starting, since 0.00558538s) 2024-09-10T18:29:59.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[41073]: from='mgr.? 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T18:29:59.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[41073]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T18:29:59.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[41073]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:29:59.737 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[41073]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:29:59.738 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[41073]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:29:59.738 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[41073]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:29:59.738 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[41073]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:29:59.738 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[41073]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:29:59.738 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[41073]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:29:59.738 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[41073]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:29:59.739 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[41073]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:29:59.739 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[41073]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:29:59.739 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[41073]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:29:59.739 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[41073]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:29:59.739 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[41073]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:29:59.739 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[41073]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:29:59.739 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[41073]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:29:59.739 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[41073]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:29:59.739 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[41073]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T18:29:59.739 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[41073]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:29:59.739 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[29603]: Active manager daemon y restarted 2024-09-10T18:29:59.739 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[29603]: Activating manager daemon y 2024-09-10T18:29:59.740 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[29603]: osdmap e66: 8 total, 8 up, 8 in 2024-09-10T18:29:59.740 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[29603]: mgrmap e28: y(active, starting, since 0.00558538s) 2024-09-10T18:29:59.740 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[29603]: from='mgr.? 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T18:29:59.740 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[29603]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T18:29:59.740 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[29603]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:29:59.740 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[29603]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:29:59.740 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[29603]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:29:59.740 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[29603]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:29:59.741 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[29603]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:29:59.741 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[29603]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:29:59.741 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[29603]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:29:59.741 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[29603]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:29:59.741 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[29603]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:29:59.742 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[29603]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:29:59.742 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[29603]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:29:59.742 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[29603]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:29:59.742 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[29603]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:29:59.742 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[29603]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:29:59.742 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[29603]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:29:59.742 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[29603]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:29:59.742 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[29603]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T18:29:59.742 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-mon[29603]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:29:59.767 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:59 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:59.467+0000 7f9bad94aec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:29:59.767 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:59 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:59.581+0000 7f9bad94aec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:29:59.767 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:59 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:59.705+0000 7f9bad94aec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:29:59.767 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:59 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:59.764+0000 7f9bad94aec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:29:59.768 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:59 smithi150 ceph-mon[32761]: Active manager daemon y restarted 2024-09-10T18:29:59.768 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:59 smithi150 ceph-mon[32761]: Activating manager daemon y 2024-09-10T18:29:59.768 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:59 smithi150 ceph-mon[32761]: osdmap e66: 8 total, 8 up, 8 in 2024-09-10T18:29:59.768 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:59 smithi150 ceph-mon[32761]: mgrmap e28: y(active, starting, since 0.00558538s) 2024-09-10T18:29:59.768 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:59 smithi150 ceph-mon[32761]: from='mgr.? 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T18:29:59.768 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:59 smithi150 ceph-mon[32761]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T18:29:59.768 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:59 smithi150 ceph-mon[32761]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:29:59.768 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:59 smithi150 ceph-mon[32761]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:29:59.768 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:59 smithi150 ceph-mon[32761]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:29:59.768 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:59 smithi150 ceph-mon[32761]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:29:59.768 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:59 smithi150 ceph-mon[32761]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:29:59.769 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:59 smithi150 ceph-mon[32761]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:29:59.769 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:59 smithi150 ceph-mon[32761]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:29:59.769 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:59 smithi150 ceph-mon[32761]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:29:59.769 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:59 smithi150 ceph-mon[32761]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:29:59.769 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:59 smithi150 ceph-mon[32761]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:29:59.769 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:59 smithi150 ceph-mon[32761]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:29:59.769 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:59 smithi150 ceph-mon[32761]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:29:59.769 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:59 smithi150 ceph-mon[32761]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:29:59.769 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:59 smithi150 ceph-mon[32761]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:29:59.769 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:59 smithi150 ceph-mon[32761]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:29:59.769 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:59 smithi150 ceph-mon[32761]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:29:59.769 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:59 smithi150 ceph-mon[32761]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T18:29:59.769 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:29:59 smithi150 ceph-mon[32761]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:29:59.908 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAjkOBmVeeGEhAANpfeeGgY/YEz7WVxuj9HVQ== 2024-09-10T18:29:59.908 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBpjeBmS0Q8JBAA43UXuB8ps4Yo5rHYcmJapA== == AQAjkOBmVeeGEhAANpfeeGgY/YEz7WVxuj9HVQ== ']' 2024-09-10T18:30:00.021 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:59 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:59.892+0000 7f9bad94aec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:30:00.021 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:29:59 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:29:59.959+0000 7f9bad94aec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:30:00.056 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: [10/Sep/2024:18:29:59] ENGINE Bus STARTING 2024-09-10T18:30:00.056 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: [10/Sep/2024:18:29:59] ENGINE Serving on http://:::9283 2024-09-10T18:30:00.057 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:29:59 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: [10/Sep/2024:18:29:59] ENGINE Bus STARTED 2024-09-10T18:30:00.305 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:30:00 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:30:00.018+0000 7f9bad94aec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:30:00.305 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:30:00 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:30:00.163+0000 7f9bad94aec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:30:00.305 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:30:00 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:30:00.232+0000 7f9bad94aec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:30:00.641 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-10T18:30:00.651 INFO:tasks.cephadm:Teardown begin 2024-09-10T18:30:00.652 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:30:00.670 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:30:00 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:30:00.302+0000 7f9bad94aec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:30:00.671 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:30:00 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:30:00.395+0000 7f9bad94aec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:30:00.671 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:30:00 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:30:00.460+0000 7f9bad94aec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:30:00.671 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:30:00 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:30:00.527+0000 7f9bad94aec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:30:00.685 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:30:00.716 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-10T18:30:00.716 DEBUG:teuthology.orchestra.run.smithi115:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T18:30:00.740 DEBUG:teuthology.orchestra.run.smithi150:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T18:30:00.773 INFO:tasks.cephadm:Stopping all daemons... 2024-09-10T18:30:00.773 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-10T18:30:00.773 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-a600e180-6fa0-11ef-bcea-c7b262605968@mon.a 2024-09-10T18:30:00.886 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-mon[41073]: Manager daemon y is now available 2024-09-10T18:30:00.887 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-mon[41073]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:30:00.887 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-mon[41073]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:30:00.887 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-mon[41073]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:30:00.887 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-mon[41073]: from='mgr.25307 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:30:00.887 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-mon[41073]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:30:00.887 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-mon[41073]: from='mgr.25307 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:30:00.887 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-mon[41073]: from='client.? 172.21.15.115:0/2203945285' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-10T18:30:00.887 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-mon[41073]: overall HEALTH_OK 2024-09-10T18:30:00.887 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-c[41050]: 2024-09-10T18:30:00.683+0000 7f6fd905b640 -1 mon.c@2(peon) e3 handle_auth_bad_method hmm, they didn't like 2 result (13) Permission denied 2024-09-10T18:30:00.887 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-c[41050]: 2024-09-10T18:30:00.884+0000 7f6fd905b640 -1 mon.c@2(peon) e3 handle_auth_bad_method hmm, they didn't like 2 result (13) Permission denied 2024-09-10T18:30:00.888 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-mon[29603]: Manager daemon y is now available 2024-09-10T18:30:00.888 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-mon[29603]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:30:00.888 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-mon[29603]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:30:00.888 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-mon[29603]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:30:00.888 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-mon[29603]: from='mgr.25307 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:30:00.888 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-mon[29603]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:30:00.888 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-mon[29603]: from='mgr.25307 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:30:00.888 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-mon[29603]: from='client.? 172.21.15.115:0/2203945285' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-10T18:30:00.888 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-mon[29603]: overall HEALTH_OK 2024-09-10T18:30:00.888 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-a[29580]: 2024-09-10T18:30:00.684+0000 7f32970ea640 -1 mon.a@0(leader) e3 handle_auth_bad_method hmm, they didn't like 2 result (13) Permission denied 2024-09-10T18:30:00.888 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:30:00 smithi115 systemd[1]: Stopping Ceph mon.a for a600e180-6fa0-11ef-bcea-c7b262605968... 2024-09-10T18:30:00.981 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:30:00 smithi150 ceph-mon[32761]: Manager daemon y is now available 2024-09-10T18:30:00.981 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:30:00 smithi150 ceph-mon[32761]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:30:00.981 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:30:00 smithi150 ceph-mon[32761]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:30:00.981 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:30:00 smithi150 ceph-mon[32761]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:30:00.981 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:30:00 smithi150 ceph-mon[32761]: from='mgr.25307 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:30:00.981 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:30:00 smithi150 ceph-mon[32761]: from='mgr.25307 172.21.15.115:0/4180900022' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:30:00.981 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:30:00 smithi150 ceph-mon[32761]: from='mgr.25307 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:30:00.981 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:30:00 smithi150 ceph-mon[32761]: from='client.? 172.21.15.115:0/2203945285' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-10T18:30:00.981 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:30:00 smithi150 ceph-mon[32761]: overall HEALTH_OK 2024-09-10T18:30:00.982 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:30:00 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-b[32725]: 2024-09-10T18:30:00.682+0000 7f266fff9640 -1 mon.b@1(peon) e3 handle_auth_bad_method hmm, they didn't like 2 result (13) Permission denied 2024-09-10T18:30:00.982 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:30:00 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-b[32725]: 2024-09-10T18:30:00.883+0000 7f266fff9640 -1 mon.b@1(peon) e3 handle_auth_bad_method hmm, they didn't like 2 result (13) Permission denied 2024-09-10T18:30:01.184 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-a[29580]: 2024-09-10T18:30:00.885+0000 7f32970ea640 -1 mon.a@0(leader) e3 handle_auth_bad_method hmm, they didn't like 2 result (13) Permission denied 2024-09-10T18:30:01.185 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: 2024-09-10T18:30:00.888+0000 7fce6b15e640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:18:30:00] ENGINE Error in HTTPServer.serve 2024-09-10T18:30:01.185 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: Traceback (most recent call last): 2024-09-10T18:30:01.185 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:30:01.185 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: self._connections.run(self.expiration_interval) 2024-09-10T18:30:01.185 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:30:01.185 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: self._run(expiration_interval) 2024-09-10T18:30:01.185 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:30:01.186 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:30:01.186 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:30:01.186 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:30:01.186 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:30:01.186 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: s = self.context.wrap_socket( 2024-09-10T18:30:01.186 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:30:01.186 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: return self.sslsocket_class._create( 2024-09-10T18:30:01.186 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:30:01.186 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: self.do_handshake() 2024-09-10T18:30:01.186 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:30:01.186 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: self._sslobj.do_handshake() 2024-09-10T18:30:01.186 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:30:01.186 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:30:00 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: 2024-09-10T18:30:01.555 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:30:01 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug there is no tcmu-runner data available 2024-09-10T18:30:01.555 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:30:01 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:30:01.315+0000 7f9bad94aec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:30:01.555 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:30:01 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: [10/Sep/2024:18:30:01] ENGINE Bus STARTING 2024-09-10T18:30:01.555 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:30:01 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: CherryPy Checker: 2024-09-10T18:30:01.555 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:30:01 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: The Application mounted at '' has an empty config. 2024-09-10T18:30:01.555 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:30:01 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: 2024-09-10T18:30:01.555 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:30:01 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: [10/Sep/2024:18:30:01] ENGINE Serving on http://:::9283 2024-09-10T18:30:01.556 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:30:01 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x[34815]: [10/Sep/2024:18:30:01] ENGINE Bus STARTED 2024-09-10T18:30:01.943 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:30:01 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-a[29580]: 2024-09-10T18:30:01.617+0000 7f329e0f8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:30:01.943 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:30:01 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-a[29580]: 2024-09-10T18:30:01.617+0000 7f329e0f8640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-10T18:30:01.943 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:30:01 smithi115 podman[87065]: 2024-09-10 18:30:01.67944366 +0000 UTC m=+0.173557226 container died b87dedb747644098ed246821a51bc3b45835b447bd981951c343671da11a7e3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-10T18:30:02.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:30:02 smithi115 podman[87065]: 2024-09-10 18:30:02.040529837 +0000 UTC m=+0.534643407 container cleanup b87dedb747644098ed246821a51bc3b45835b447bd981951c343671da11a7e3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-10T18:30:02.235 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 18:30:02 smithi115 bash[87065]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-a 2024-09-10T18:30:02.492 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a600e180-6fa0-11ef-bcea-c7b262605968@mon.a.service' 2024-09-10T18:30:02.559 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:30:02.559 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-10T18:30:02.559 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-09-10T18:30:02.559 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-a600e180-6fa0-11ef-bcea-c7b262605968@mon.c 2024-09-10T18:30:02.921 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:30:02 smithi115 systemd[1]: Stopping Ceph mon.c for a600e180-6fa0-11ef-bcea-c7b262605968... 2024-09-10T18:30:03.735 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:30:03 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: [10/Sep/2024:18:30:03] ENGINE Bus STOPPING 2024-09-10T18:30:03.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:30:03 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-c[41050]: 2024-09-10T18:30:03.476+0000 7f6fda85e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:30:03.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:30:03 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-c[41050]: 2024-09-10T18:30:03.476+0000 7f6fda85e640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-09-10T18:30:04.110 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:30:03 smithi115 podman[87397]: 2024-09-10 18:30:03.788735087 +0000 UTC m=+0.750909570 container died fa822934dbab3984321369afe7084bb1560637c7e78ecf8053d45112bf59b548 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-10T18:30:04.111 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:30:03 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: [10/Sep/2024:18:30:03] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:30:04.111 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:30:03 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: [10/Sep/2024:18:30:03] ENGINE Bus STOPPED 2024-09-10T18:30:04.111 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:30:03 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: [10/Sep/2024:18:30:03] ENGINE Bus STARTING 2024-09-10T18:30:04.111 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:30:04 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: [10/Sep/2024:18:30:04] ENGINE Serving on http://:::9283 2024-09-10T18:30:04.111 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:30:04 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: [10/Sep/2024:18:30:04] ENGINE Bus STARTED 2024-09-10T18:30:04.411 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:30:04 smithi115 podman[87397]: 2024-09-10 18:30:04.217080178 +0000 UTC m=+1.179254660 container cleanup fa822934dbab3984321369afe7084bb1560637c7e78ecf8053d45112bf59b548 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-c, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-10T18:30:04.411 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:30:04 smithi115 bash[87397]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-c 2024-09-10T18:30:04.679 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 18:30:04 smithi115 podman[87462]: 2024-09-10 18:30:04.41012235 +0000 UTC m=+0.740749288 container remove fa822934dbab3984321369afe7084bb1560637c7e78ecf8053d45112bf59b548 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:30:04.965 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:30:04 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: ::ffff:172.21.15.150 - - [10/Sep/2024:18:30:04] "GET /metrics HTTP/1.1" 200 34933 "" "Prometheus/2.43.0" 2024-09-10T18:30:04.978 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a600e180-6fa0-11ef-bcea-c7b262605968@mon.c.service' 2024-09-10T18:30:05.016 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:30:05.016 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-09-10T18:30:05.016 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-09-10T18:30:05.016 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-a600e180-6fa0-11ef-bcea-c7b262605968@mon.b 2024-09-10T18:30:05.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:30:05 smithi150 systemd[1]: Stopping Ceph mon.b for a600e180-6fa0-11ef-bcea-c7b262605968... 2024-09-10T18:30:05.742 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:30:05 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-b[32725]: 2024-09-10T18:30:05.322+0000 7f2677007640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:30:05.742 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:30:05 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-b[32725]: 2024-09-10T18:30:05.322+0000 7f2677007640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-09-10T18:30:05.742 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:30:05 smithi150 podman[73141]: 2024-09-10 18:30:05.436384272 +0000 UTC m=+0.220254172 container died bb0215e57dbdd1ee204189c64c28d1f943bebb295ef3a8aa0c38aaea2365a4ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0) 2024-09-10T18:30:06.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:30:05 smithi150 podman[73141]: 2024-09-10 18:30:05.831006708 +0000 UTC m=+0.614876598 container cleanup bb0215e57dbdd1ee204189c64c28d1f943bebb295ef3a8aa0c38aaea2365a4ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-b, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-10T18:30:06.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 18:30:05 smithi150 bash[73141]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mon-b 2024-09-10T18:30:06.266 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a600e180-6fa0-11ef-bcea-c7b262605968@mon.b.service' 2024-09-10T18:30:06.320 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:30:06.321 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-09-10T18:30:06.321 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-10T18:30:06.321 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-a600e180-6fa0-11ef-bcea-c7b262605968@mgr.y 2024-09-10T18:30:06.627 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:30:06 smithi115 systemd[1]: Stopping Ceph mgr.y for a600e180-6fa0-11ef-bcea-c7b262605968... 2024-09-10T18:30:06.975 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:30:06 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y[86502]: 2024-09-10T18:30:06.624+0000 7fcec4572640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:30:06.975 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:30:06 smithi115 podman[87615]: 2024-09-10 18:30:06.656050968 +0000 UTC m=+0.143844367 container died ba5505bf37f5320044656588855075cdc312c9cfd122d70105d78ec44b0af3b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T18:30:07.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:30:07 smithi115 podman[87615]: 2024-09-10 18:30:07.056394179 +0000 UTC m=+0.544187574 container cleanup ba5505bf37f5320044656588855075cdc312c9cfd122d70105d78ec44b0af3b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-10T18:30:07.235 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 18:30:07 smithi115 bash[87615]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-y 2024-09-10T18:30:07.424 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a600e180-6fa0-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-10T18:30:07.459 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:30:07.459 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-10T18:30:07.459 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-10T18:30:07.460 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-a600e180-6fa0-11ef-bcea-c7b262605968@mgr.x 2024-09-10T18:30:07.802 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:30:07 smithi150 systemd[1]: Stopping Ceph mgr.x for a600e180-6fa0-11ef-bcea-c7b262605968... 2024-09-10T18:30:08.054 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:30:07 smithi150 podman[73271]: 2024-09-10 18:30:07.80073635 +0000 UTC m=+0.148179850 container died 5f889455cd5022c28ace2a5f7eb56131b238ec0523b57a5a8ddb7c3787119c5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True) 2024-09-10T18:30:08.430 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:30:08 smithi150 podman[73271]: 2024-09-10 18:30:08.177936877 +0000 UTC m=+0.525380376 container cleanup 5f889455cd5022c28ace2a5f7eb56131b238ec0523b57a5a8ddb7c3787119c5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-10T18:30:08.430 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 18:30:08 smithi150 bash[73271]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968-mgr-x 2024-09-10T18:30:08.603 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a600e180-6fa0-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-10T18:30:08.638 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:30:08.639 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-10T18:30:08.639 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-10T18:30:08.639 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-a600e180-6fa0-11ef-bcea-c7b262605968@osd.0 2024-09-10T18:30:08.955 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 18:30:08 smithi115 systemd[1]: Stopping Ceph osd.0 for a600e180-6fa0-11ef-bcea-c7b262605968... 2024-09-10T18:30:09.234 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 18:30:08 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-0[45326]: 2024-09-10T18:30:08.953+0000 7f407fcd1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:30:09.235 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 18:30:08 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-0[45326]: 2024-09-10T18:30:08.953+0000 7f407fcd1640 -1 osd.0 66 *** Got signal Terminated *** 2024-09-10T18:30:09.235 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 18:30:08 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-0[45326]: 2024-09-10T18:30:08.953+0000 7f407fcd1640 -1 osd.0 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:30:14.712 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 18:30:14 smithi115 podman[87747]: 2024-09-10 18:30:14.450123824 +0000 UTC m=+5.613017153 container died fc8e8e7fefb692c879a59f1e2110828505577055277db00e13ab5083fbecffd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T18:30:14.985 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 18:30:14 smithi115 podman[87747]: 2024-09-10 18:30:14.792496643 +0000 UTC m=+5.955389957 container cleanup fc8e8e7fefb692c879a59f1e2110828505577055277db00e13ab5083fbecffd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-10T18:30:14.985 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 18:30:14 smithi115 bash[87747]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-0 2024-09-10T18:30:15.414 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 18:30:15 smithi115 podman[87769]: 2024-09-10 18:30:15.002104167 +0000 UTC m=+0.551041399 container remove fc8e8e7fefb692c879a59f1e2110828505577055277db00e13ab5083fbecffd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-0, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:30:15.689 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 18:30:15 smithi115 podman[87865]: 2024-09-10 18:30:15.316182341 +0000 UTC m=+0.015624979 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:30:15.690 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 18:30:15 smithi115 podman[87865]: 2024-09-10 18:30:15.427694607 +0000 UTC m=+0.127137245 container create 82838a56d620185bf1815924686cca074e739148462cb678d8f43cffc804cb88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-10T18:30:15.690 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 18:30:15 smithi115 podman[87865]: 2024-09-10 18:30:15.688452783 +0000 UTC m=+0.387895421 container init 82838a56d620185bf1815924686cca074e739148462cb678d8f43cffc804cb88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-10T18:30:15.985 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 18:30:15 smithi115 podman[87865]: 2024-09-10 18:30:15.69324415 +0000 UTC m=+0.392686791 container start 82838a56d620185bf1815924686cca074e739148462cb678d8f43cffc804cb88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-0-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS) 2024-09-10T18:30:15.985 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 18:30:15 smithi115 podman[87865]: 2024-09-10 18:30:15.788441735 +0000 UTC m=+0.487884388 container attach 82838a56d620185bf1815924686cca074e739148462cb678d8f43cffc804cb88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-0-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD) 2024-09-10T18:30:15.986 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 18:30:15 smithi115 podman[87865]: 2024-09-10 18:30:15.872824043 +0000 UTC m=+0.572266689 container died 82838a56d620185bf1815924686cca074e739148462cb678d8f43cffc804cb88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:30:16.478 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a600e180-6fa0-11ef-bcea-c7b262605968@osd.0.service' 2024-09-10T18:30:16.516 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:30:16.516 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-10T18:30:16.516 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-10T18:30:16.516 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-a600e180-6fa0-11ef-bcea-c7b262605968@osd.1 2024-09-10T18:30:16.735 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 18:30:16 smithi115 systemd[1]: Stopping Ceph osd.1 for a600e180-6fa0-11ef-bcea-c7b262605968... 2024-09-10T18:30:17.235 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 18:30:16 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-1[51767]: 2024-09-10T18:30:16.844+0000 7fef7c8ae640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:30:17.235 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 18:30:16 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-1[51767]: 2024-09-10T18:30:16.844+0000 7fef7c8ae640 -1 osd.1 66 *** Got signal Terminated *** 2024-09-10T18:30:17.235 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 18:30:16 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-1[51767]: 2024-09-10T18:30:16.844+0000 7fef7c8ae640 -1 osd.1 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:30:22.644 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 18:30:22 smithi115 podman[87991]: 2024-09-10 18:30:22.372677368 +0000 UTC m=+5.637388951 container died 39dcf7a8048f0c25631e30de7966f78a5fa45f57d1bf543ebed2852aea79c7b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-1, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T18:30:22.985 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 18:30:22 smithi115 podman[87991]: 2024-09-10 18:30:22.751671723 +0000 UTC m=+6.016383296 container cleanup 39dcf7a8048f0c25631e30de7966f78a5fa45f57d1bf543ebed2852aea79c7b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-1, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T18:30:22.985 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 18:30:22 smithi115 bash[87991]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-1 2024-09-10T18:30:23.367 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 18:30:23 smithi115 podman[88013]: 2024-09-10 18:30:23.013221085 +0000 UTC m=+0.642431918 container remove 39dcf7a8048f0c25631e30de7966f78a5fa45f57d1bf543ebed2852aea79c7b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-10T18:30:23.668 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 18:30:23 smithi115 podman[88108]: 2024-09-10 18:30:23.268209122 +0000 UTC m=+0.015467195 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:30:23.668 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 18:30:23 smithi115 podman[88108]: 2024-09-10 18:30:23.372349571 +0000 UTC m=+0.119607645 container create f2c9512f520b1e42cd9085fa9f15ed30ca4183d9b624233a36df6bc20896757f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:30:23.668 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 18:30:23 smithi115 podman[88108]: 2024-09-10 18:30:23.600030473 +0000 UTC m=+0.347288549 container init f2c9512f520b1e42cd9085fa9f15ed30ca4183d9b624233a36df6bc20896757f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2) 2024-09-10T18:30:23.668 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 18:30:23 smithi115 podman[88108]: 2024-09-10 18:30:23.60525659 +0000 UTC m=+0.352514671 container start f2c9512f520b1e42cd9085fa9f15ed30ca4183d9b624233a36df6bc20896757f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-10T18:30:23.668 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 18:30:23 smithi115 podman[88108]: 2024-09-10 18:30:23.666128858 +0000 UTC m=+0.413386944 container attach f2c9512f520b1e42cd9085fa9f15ed30ca4183d9b624233a36df6bc20896757f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-10T18:30:23.985 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 18:30:23 smithi115 podman[88108]: 2024-09-10 18:30:23.780701141 +0000 UTC m=+0.527959215 container died f2c9512f520b1e42cd9085fa9f15ed30ca4183d9b624233a36df6bc20896757f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-10T18:30:24.407 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a600e180-6fa0-11ef-bcea-c7b262605968@osd.1.service' 2024-09-10T18:30:24.447 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:30:24.447 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-10T18:30:24.447 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-10T18:30:24.448 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-a600e180-6fa0-11ef-bcea-c7b262605968@osd.2 2024-09-10T18:30:24.666 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 18:30:24 smithi115 systemd[1]: Stopping Ceph osd.2 for a600e180-6fa0-11ef-bcea-c7b262605968... 2024-09-10T18:30:24.985 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 18:30:24 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-2[58201]: 2024-09-10T18:30:24.764+0000 7f403accd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:30:24.985 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 18:30:24 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-2[58201]: 2024-09-10T18:30:24.764+0000 7f403accd640 -1 osd.2 66 *** Got signal Terminated *** 2024-09-10T18:30:24.985 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 18:30:24 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-2[58201]: 2024-09-10T18:30:24.764+0000 7f403accd640 -1 osd.2 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:30:30.408 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 18:30:30 smithi115 podman[88237]: 2024-09-10 18:30:30.124081479 +0000 UTC m=+5.455846432 container died ec0e05063b404f9ac8e028ea4e73b307cd00627e8a55291f4538d9396c548b3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-10T18:30:30.737 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 18:30:30 smithi115 podman[88237]: 2024-09-10 18:30:30.504876735 +0000 UTC m=+5.836641676 container cleanup ec0e05063b404f9ac8e028ea4e73b307cd00627e8a55291f4538d9396c548b3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-10T18:30:30.737 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 18:30:30 smithi115 bash[88237]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-2 2024-09-10T18:30:31.126 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 18:30:30 smithi115 podman[88268]: 2024-09-10 18:30:30.772561848 +0000 UTC m=+0.641523069 container remove ec0e05063b404f9ac8e028ea4e73b307cd00627e8a55291f4538d9396c548b3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-2, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T18:30:31.443 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 18:30:31 smithi115 podman[88364]: 2024-09-10 18:30:31.027521926 +0000 UTC m=+0.016159313 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:30:31.444 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 18:30:31 smithi115 podman[88364]: 2024-09-10 18:30:31.131574141 +0000 UTC m=+0.120211523 container create 137700221ba6e99f46cc9604257fdd6542df073e9f0c2d59d7e16b3759a3b09a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-10T18:30:31.444 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 18:30:31 smithi115 podman[88364]: 2024-09-10 18:30:31.350857791 +0000 UTC m=+0.339495173 container init 137700221ba6e99f46cc9604257fdd6542df073e9f0c2d59d7e16b3759a3b09a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:30:31.444 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 18:30:31 smithi115 podman[88364]: 2024-09-10 18:30:31.355540287 +0000 UTC m=+0.344177670 container start 137700221ba6e99f46cc9604257fdd6542df073e9f0c2d59d7e16b3759a3b09a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-10T18:30:31.444 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 18:30:31 smithi115 podman[88364]: 2024-09-10 18:30:31.442098464 +0000 UTC m=+0.430735846 container attach 137700221ba6e99f46cc9604257fdd6542df073e9f0c2d59d7e16b3759a3b09a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD) 2024-09-10T18:30:31.735 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 18:30:31 smithi115 podman[88364]: 2024-09-10 18:30:31.53387359 +0000 UTC m=+0.522510971 container died 137700221ba6e99f46cc9604257fdd6542df073e9f0c2d59d7e16b3759a3b09a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-2-deactivate, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:30:32.180 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a600e180-6fa0-11ef-bcea-c7b262605968@osd.2.service' 2024-09-10T18:30:32.220 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:30:32.220 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-10T18:30:32.220 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-10T18:30:32.221 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-a600e180-6fa0-11ef-bcea-c7b262605968@osd.3 2024-09-10T18:30:32.485 INFO:journalctl@ceph.osd.3.smithi115.stdout:Sep 10 18:30:32 smithi115 systemd[1]: Stopping Ceph osd.3 for a600e180-6fa0-11ef-bcea-c7b262605968... 2024-09-10T18:30:32.985 INFO:journalctl@ceph.osd.3.smithi115.stdout:Sep 10 18:30:32 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-3[64304]: 2024-09-10T18:30:32.542+0000 7fad728a1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:30:32.985 INFO:journalctl@ceph.osd.3.smithi115.stdout:Sep 10 18:30:32 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-3[64304]: 2024-09-10T18:30:32.542+0000 7fad728a1640 -1 osd.3 66 *** Got signal Terminated *** 2024-09-10T18:30:32.985 INFO:journalctl@ceph.osd.3.smithi115.stdout:Sep 10 18:30:32 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-3[64304]: 2024-09-10T18:30:32.542+0000 7fad728a1640 -1 osd.3 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:30:35.305 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:34 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5[44072]: 2024-09-10T18:30:34.828+0000 7fbd9d5be640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.198315+0000 front 2024-09-10T18:30:09.198366+0000 (oldest deadline 2024-09-10T18:30:34.498606+0000) 2024-09-10T18:30:35.485 INFO:journalctl@ceph.osd.3.smithi115.stdout:Sep 10 18:30:35 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-3[64304]: 2024-09-10T18:30:35.247+0000 7fad706bc640 -1 osd.3 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:08.993386+0000 front 2024-09-10T18:30:08.993414+0000 (oldest deadline 2024-09-10T18:30:34.893587+0000) 2024-09-10T18:30:35.831 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:35 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5[44072]: 2024-09-10T18:30:35.806+0000 7fbd9d5be640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.198315+0000 front 2024-09-10T18:30:09.198366+0000 (oldest deadline 2024-09-10T18:30:34.498606+0000) 2024-09-10T18:30:36.305 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:35 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:35.828+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.585731+0000 front 2024-09-10T18:30:09.585783+0000 (oldest deadline 2024-09-10T18:30:35.485625+0000) 2024-09-10T18:30:36.485 INFO:journalctl@ceph.osd.3.smithi115.stdout:Sep 10 18:30:36 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-3[64304]: 2024-09-10T18:30:36.207+0000 7fad706bc640 -1 osd.3 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:08.993386+0000 front 2024-09-10T18:30:08.993414+0000 (oldest deadline 2024-09-10T18:30:34.893587+0000) 2024-09-10T18:30:37.305 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:36 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5[44072]: 2024-09-10T18:30:36.849+0000 7fbd9d5be640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.198315+0000 front 2024-09-10T18:30:09.198366+0000 (oldest deadline 2024-09-10T18:30:34.498606+0000) 2024-09-10T18:30:37.305 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:36 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:36.807+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.585731+0000 front 2024-09-10T18:30:09.585783+0000 (oldest deadline 2024-09-10T18:30:35.485625+0000) 2024-09-10T18:30:37.484 INFO:journalctl@ceph.osd.3.smithi115.stdout:Sep 10 18:30:37 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-3[64304]: 2024-09-10T18:30:37.171+0000 7fad706bc640 -1 osd.3 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:08.993386+0000 front 2024-09-10T18:30:08.993414+0000 (oldest deadline 2024-09-10T18:30:34.893587+0000) 2024-09-10T18:30:38.055 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:37 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5[44072]: 2024-09-10T18:30:37.807+0000 7fbd9d5be640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.198315+0000 front 2024-09-10T18:30:09.198366+0000 (oldest deadline 2024-09-10T18:30:34.498606+0000) 2024-09-10T18:30:38.055 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:37 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:37.791+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.585731+0000 front 2024-09-10T18:30:09.585783+0000 (oldest deadline 2024-09-10T18:30:35.485625+0000) 2024-09-10T18:30:38.163 INFO:journalctl@ceph.osd.3.smithi115.stdout:Sep 10 18:30:37 smithi115 podman[88491]: 2024-09-10 18:30:37.868476619 +0000 UTC m=+5.434947798 container died 61e6b95a1524e4d12b2d29e71b060fb52eb17c2e16643bdc3a41bb1716fd9cab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-3, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-10T18:30:38.485 INFO:journalctl@ceph.osd.3.smithi115.stdout:Sep 10 18:30:38 smithi115 podman[88491]: 2024-09-10 18:30:38.247179764 +0000 UTC m=+5.813650935 container cleanup 61e6b95a1524e4d12b2d29e71b060fb52eb17c2e16643bdc3a41bb1716fd9cab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS) 2024-09-10T18:30:38.485 INFO:journalctl@ceph.osd.3.smithi115.stdout:Sep 10 18:30:38 smithi115 bash[88491]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-3 2024-09-10T18:30:38.796 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:38 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:38.383+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:08.860760+0000 front 2024-09-10T18:30:08.860785+0000 (oldest deadline 2024-09-10T18:30:38.261217+0000) 2024-09-10T18:30:38.812 INFO:journalctl@ceph.osd.3.smithi115.stdout:Sep 10 18:30:38 smithi115 podman[88512]: 2024-09-10 18:30:38.489922826 +0000 UTC m=+0.617990814 container remove 61e6b95a1524e4d12b2d29e71b060fb52eb17c2e16643bdc3a41bb1716fd9cab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-3, ceph=True, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-10T18:30:39.055 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:38 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5[44072]: 2024-09-10T18:30:38.840+0000 7fbd9d5be640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.198315+0000 front 2024-09-10T18:30:09.198366+0000 (oldest deadline 2024-09-10T18:30:34.498606+0000) 2024-09-10T18:30:39.055 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:38 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:38.794+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.585731+0000 front 2024-09-10T18:30:09.585783+0000 (oldest deadline 2024-09-10T18:30:35.485625+0000) 2024-09-10T18:30:39.128 INFO:journalctl@ceph.osd.3.smithi115.stdout:Sep 10 18:30:38 smithi115 podman[88607]: 2024-09-10 18:30:38.715009375 +0000 UTC m=+0.015759162 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:30:39.128 INFO:journalctl@ceph.osd.3.smithi115.stdout:Sep 10 18:30:38 smithi115 podman[88607]: 2024-09-10 18:30:38.823919008 +0000 UTC m=+0.124668784 container create 93d7d3a9c09c4c5c3d321bef8f56bf588de197bfa4daac9163519dc2ba03141d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-10T18:30:39.128 INFO:journalctl@ceph.osd.3.smithi115.stdout:Sep 10 18:30:39 smithi115 podman[88607]: 2024-09-10 18:30:39.034465013 +0000 UTC m=+0.335214787 container init 93d7d3a9c09c4c5c3d321bef8f56bf588de197bfa4daac9163519dc2ba03141d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-10T18:30:39.128 INFO:journalctl@ceph.osd.3.smithi115.stdout:Sep 10 18:30:39 smithi115 podman[88607]: 2024-09-10 18:30:39.039327399 +0000 UTC m=+0.340077174 container start 93d7d3a9c09c4c5c3d321bef8f56bf588de197bfa4daac9163519dc2ba03141d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-10T18:30:39.128 INFO:journalctl@ceph.osd.3.smithi115.stdout:Sep 10 18:30:39 smithi115 podman[88607]: 2024-09-10 18:30:39.12587097 +0000 UTC m=+0.426620745 container attach 93d7d3a9c09c4c5c3d321bef8f56bf588de197bfa4daac9163519dc2ba03141d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-10T18:30:39.485 INFO:journalctl@ceph.osd.3.smithi115.stdout:Sep 10 18:30:39 smithi115 podman[88607]: 2024-09-10 18:30:39.21114545 +0000 UTC m=+0.511895239 container died 93d7d3a9c09c4c5c3d321bef8f56bf588de197bfa4daac9163519dc2ba03141d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-3-deactivate, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:30:39.777 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:39 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:39.376+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:08.860760+0000 front 2024-09-10T18:30:08.860785+0000 (oldest deadline 2024-09-10T18:30:38.261217+0000) 2024-09-10T18:30:39.824 INFO:journalctl@ceph.osd.3.smithi115.stdout:Sep 10 18:30:39 smithi115 podman[88649]: 2024-09-10 18:30:39.820932316 +0000 UTC m=+0.603466473 container remove 93d7d3a9c09c4c5c3d321bef8f56bf588de197bfa4daac9163519dc2ba03141d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-09-10T18:30:39.845 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a600e180-6fa0-11ef-bcea-c7b262605968@osd.3.service' 2024-09-10T18:30:39.886 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:30:39.886 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-10T18:30:39.886 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-10T18:30:39.887 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-a600e180-6fa0-11ef-bcea-c7b262605968@osd.4 2024-09-10T18:30:40.055 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 18:30:39 smithi150 systemd[1]: Stopping Ceph osd.4 for a600e180-6fa0-11ef-bcea-c7b262605968... 2024-09-10T18:30:40.055 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:39 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5[44072]: 2024-09-10T18:30:39.813+0000 7fbd9d5be640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.198315+0000 front 2024-09-10T18:30:09.198366+0000 (oldest deadline 2024-09-10T18:30:34.498606+0000) 2024-09-10T18:30:40.056 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:39 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:39.775+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.585731+0000 front 2024-09-10T18:30:09.585783+0000 (oldest deadline 2024-09-10T18:30:35.485625+0000) 2024-09-10T18:30:40.556 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:40 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:40.353+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:08.860760+0000 front 2024-09-10T18:30:08.860785+0000 (oldest deadline 2024-09-10T18:30:38.261217+0000) 2024-09-10T18:30:40.556 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 18:30:40 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-4[38186]: 2024-09-10T18:30:40.197+0000 7f9fd8c95640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:30:40.556 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 18:30:40 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-4[38186]: 2024-09-10T18:30:40.197+0000 7f9fd8c95640 -1 osd.4 66 *** Got signal Terminated *** 2024-09-10T18:30:40.556 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 18:30:40 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-4[38186]: 2024-09-10T18:30:40.197+0000 7f9fd8c95640 -1 osd.4 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:30:40.556 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 18:30:40 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-4[38186]: 2024-09-10T18:30:40.228+0000 7f9fd22a7640 -1 osd.4 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:11.064906+0000 front 2024-09-10T18:30:11.064954+0000 (oldest deadline 2024-09-10T18:30:39.265225+0000) 2024-09-10T18:30:41.055 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:40 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5[44072]: 2024-09-10T18:30:40.845+0000 7fbd9d5be640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.198315+0000 front 2024-09-10T18:30:09.198366+0000 (oldest deadline 2024-09-10T18:30:34.498606+0000) 2024-09-10T18:30:41.055 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:40 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:40.747+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.585731+0000 front 2024-09-10T18:30:09.585783+0000 (oldest deadline 2024-09-10T18:30:35.485625+0000) 2024-09-10T18:30:41.555 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 18:30:41 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-4[38186]: 2024-09-10T18:30:41.266+0000 7f9fd22a7640 -1 osd.4 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:11.064906+0000 front 2024-09-10T18:30:11.064954+0000 (oldest deadline 2024-09-10T18:30:39.265225+0000) 2024-09-10T18:30:41.555 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:41 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:41.398+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:08.860760+0000 front 2024-09-10T18:30:08.860785+0000 (oldest deadline 2024-09-10T18:30:38.261217+0000) 2024-09-10T18:30:42.055 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:41 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:41.795+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.585731+0000 front 2024-09-10T18:30:09.585783+0000 (oldest deadline 2024-09-10T18:30:35.485625+0000) 2024-09-10T18:30:42.055 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:41 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5[44072]: 2024-09-10T18:30:41.824+0000 7fbd9d5be640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.198315+0000 front 2024-09-10T18:30:09.198366+0000 (oldest deadline 2024-09-10T18:30:34.498606+0000) 2024-09-10T18:30:42.554 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 18:30:42 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-4[38186]: 2024-09-10T18:30:42.284+0000 7f9fd22a7640 -1 osd.4 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:11.064906+0000 front 2024-09-10T18:30:11.064954+0000 (oldest deadline 2024-09-10T18:30:39.265225+0000) 2024-09-10T18:30:42.555 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:42 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:42.444+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:08.860760+0000 front 2024-09-10T18:30:08.860785+0000 (oldest deadline 2024-09-10T18:30:38.261217+0000) 2024-09-10T18:30:43.055 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:42 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:42.763+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.585731+0000 front 2024-09-10T18:30:09.585783+0000 (oldest deadline 2024-09-10T18:30:35.485625+0000) 2024-09-10T18:30:43.055 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:42 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5[44072]: 2024-09-10T18:30:42.794+0000 7fbd9d5be640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.198315+0000 front 2024-09-10T18:30:09.198366+0000 (oldest deadline 2024-09-10T18:30:34.498606+0000) 2024-09-10T18:30:43.555 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:43 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:43.468+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:08.860760+0000 front 2024-09-10T18:30:08.860785+0000 (oldest deadline 2024-09-10T18:30:38.261217+0000) 2024-09-10T18:30:43.616 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 18:30:43 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-4[38186]: 2024-09-10T18:30:43.262+0000 7f9fd22a7640 -1 osd.4 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:11.064906+0000 front 2024-09-10T18:30:11.064954+0000 (oldest deadline 2024-09-10T18:30:39.265225+0000) 2024-09-10T18:30:43.616 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 18:30:43 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-4[38186]: 2024-09-10T18:30:43.262+0000 7f9fd22a7640 -1 osd.4 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:19.265663+0000 front 2024-09-10T18:30:19.265586+0000 (oldest deadline 2024-09-10T18:30:42.765688+0000) 2024-09-10T18:30:44.055 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:43 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:43.760+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.585731+0000 front 2024-09-10T18:30:09.585783+0000 (oldest deadline 2024-09-10T18:30:35.485625+0000) 2024-09-10T18:30:44.055 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:43 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5[44072]: 2024-09-10T18:30:43.778+0000 7fbd9d5be640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.198315+0000 front 2024-09-10T18:30:09.198366+0000 (oldest deadline 2024-09-10T18:30:34.498606+0000) 2024-09-10T18:30:44.055 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:43 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5[44072]: 2024-09-10T18:30:43.778+0000 7fbd9d5be640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:20.399544+0000 front 2024-09-10T18:30:20.399636+0000 (oldest deadline 2024-09-10T18:30:43.200044+0000) 2024-09-10T18:30:44.555 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 18:30:44 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-4[38186]: 2024-09-10T18:30:44.282+0000 7f9fd22a7640 -1 osd.4 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:11.064906+0000 front 2024-09-10T18:30:11.064954+0000 (oldest deadline 2024-09-10T18:30:39.265225+0000) 2024-09-10T18:30:44.555 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 18:30:44 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-4[38186]: 2024-09-10T18:30:44.282+0000 7f9fd22a7640 -1 osd.4 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:19.265663+0000 front 2024-09-10T18:30:19.265586+0000 (oldest deadline 2024-09-10T18:30:42.765688+0000) 2024-09-10T18:30:44.555 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:44 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:44.489+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:08.860760+0000 front 2024-09-10T18:30:08.860785+0000 (oldest deadline 2024-09-10T18:30:38.261217+0000) 2024-09-10T18:30:44.555 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:44 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:44.489+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.762380+0000 front 2024-09-10T18:30:21.762329+0000 (oldest deadline 2024-09-10T18:30:44.062166+0000) 2024-09-10T18:30:45.055 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:44 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:44.747+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.585731+0000 front 2024-09-10T18:30:09.585783+0000 (oldest deadline 2024-09-10T18:30:35.485625+0000) 2024-09-10T18:30:45.055 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:44 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5[44072]: 2024-09-10T18:30:44.775+0000 7fbd9d5be640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.198315+0000 front 2024-09-10T18:30:09.198366+0000 (oldest deadline 2024-09-10T18:30:34.498606+0000) 2024-09-10T18:30:45.055 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:44 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5[44072]: 2024-09-10T18:30:44.775+0000 7fbd9d5be640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:20.399544+0000 front 2024-09-10T18:30:20.399636+0000 (oldest deadline 2024-09-10T18:30:43.200044+0000) 2024-09-10T18:30:45.733 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:45 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:45.462+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:08.860760+0000 front 2024-09-10T18:30:08.860785+0000 (oldest deadline 2024-09-10T18:30:38.261217+0000) 2024-09-10T18:30:45.734 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:45 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:45.462+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.762380+0000 front 2024-09-10T18:30:21.762329+0000 (oldest deadline 2024-09-10T18:30:44.062166+0000) 2024-09-10T18:30:46.041 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 18:30:45 smithi150 podman[73417]: 2024-09-10 18:30:45.732227198 +0000 UTC m=+5.650420117 container died e60a82dbe5d4eb09b74681d6d61dd497e47d58960009e905ca767fe2de610854 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-4, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-10T18:30:46.041 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:45 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:45.741+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.585731+0000 front 2024-09-10T18:30:09.585783+0000 (oldest deadline 2024-09-10T18:30:35.485625+0000) 2024-09-10T18:30:46.041 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:45 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:45.741+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.386848+0000 front 2024-09-10T18:30:21.386832+0000 (oldest deadline 2024-09-10T18:30:44.886674+0000) 2024-09-10T18:30:46.042 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:45 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5[44072]: 2024-09-10T18:30:45.805+0000 7fbd9d5be640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.198315+0000 front 2024-09-10T18:30:09.198366+0000 (oldest deadline 2024-09-10T18:30:34.498606+0000) 2024-09-10T18:30:46.042 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:45 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5[44072]: 2024-09-10T18:30:45.805+0000 7fbd9d5be640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:20.399544+0000 front 2024-09-10T18:30:20.399636+0000 (oldest deadline 2024-09-10T18:30:43.200044+0000) 2024-09-10T18:30:46.305 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 18:30:46 smithi150 podman[73417]: 2024-09-10 18:30:46.130247292 +0000 UTC m=+6.048440215 container cleanup e60a82dbe5d4eb09b74681d6d61dd497e47d58960009e905ca767fe2de610854 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-10T18:30:46.305 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 18:30:46 smithi150 bash[73417]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-4 2024-09-10T18:30:46.305 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 18:30:46 smithi150 podman[73438]: 2024-09-10 18:30:46.289917498 +0000 UTC m=+0.549800810 container remove e60a82dbe5d4eb09b74681d6d61dd497e47d58960009e905ca767fe2de610854 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-4, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-10T18:30:46.719 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 18:30:46 smithi150 podman[73532]: 2024-09-10 18:30:46.613200888 +0000 UTC m=+0.016329889 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:30:46.719 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:46 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:46.461+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:08.860760+0000 front 2024-09-10T18:30:08.860785+0000 (oldest deadline 2024-09-10T18:30:38.261217+0000) 2024-09-10T18:30:46.719 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:46 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:46.461+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.762380+0000 front 2024-09-10T18:30:21.762329+0000 (oldest deadline 2024-09-10T18:30:44.062166+0000) 2024-09-10T18:30:47.035 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:46 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5[44072]: 2024-09-10T18:30:46.765+0000 7fbd9d5be640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.198315+0000 front 2024-09-10T18:30:09.198366+0000 (oldest deadline 2024-09-10T18:30:34.498606+0000) 2024-09-10T18:30:47.036 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:46 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5[44072]: 2024-09-10T18:30:46.765+0000 7fbd9d5be640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:20.399544+0000 front 2024-09-10T18:30:20.399636+0000 (oldest deadline 2024-09-10T18:30:43.200044+0000) 2024-09-10T18:30:47.036 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:46 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:46.716+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.585731+0000 front 2024-09-10T18:30:09.585783+0000 (oldest deadline 2024-09-10T18:30:35.485625+0000) 2024-09-10T18:30:47.036 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:46 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:46.716+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.386848+0000 front 2024-09-10T18:30:21.386832+0000 (oldest deadline 2024-09-10T18:30:44.886674+0000) 2024-09-10T18:30:47.036 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 18:30:46 smithi150 podman[73532]: 2024-09-10 18:30:46.732450633 +0000 UTC m=+0.135579620 container create 08879aca71916e5731cb9d4b59503cb0a3d33372791f4244d52cfb05c4a907b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-10T18:30:47.036 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 18:30:46 smithi150 podman[73532]: 2024-09-10 18:30:46.943218288 +0000 UTC m=+0.346347276 container init 08879aca71916e5731cb9d4b59503cb0a3d33372791f4244d52cfb05c4a907b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-10T18:30:47.037 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 18:30:46 smithi150 podman[73532]: 2024-09-10 18:30:46.948413252 +0000 UTC m=+0.351542241 container start 08879aca71916e5731cb9d4b59503cb0a3d33372791f4244d52cfb05c4a907b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731) 2024-09-10T18:30:47.037 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 18:30:47 smithi150 podman[73532]: 2024-09-10 18:30:47.03471578 +0000 UTC m=+0.437844774 container attach 08879aca71916e5731cb9d4b59503cb0a3d33372791f4244d52cfb05c4a907b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-10T18:30:47.305 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 18:30:47 smithi150 podman[73532]: 2024-09-10 18:30:47.117017242 +0000 UTC m=+0.520146243 container died 08879aca71916e5731cb9d4b59503cb0a3d33372791f4244d52cfb05c4a907b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-10T18:30:47.716 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:47 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:47.453+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:08.860760+0000 front 2024-09-10T18:30:08.860785+0000 (oldest deadline 2024-09-10T18:30:38.261217+0000) 2024-09-10T18:30:47.716 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:47 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:47.453+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.762380+0000 front 2024-09-10T18:30:21.762329+0000 (oldest deadline 2024-09-10T18:30:44.062166+0000) 2024-09-10T18:30:47.733 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a600e180-6fa0-11ef-bcea-c7b262605968@osd.4.service' 2024-09-10T18:30:47.773 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:30:47.773 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-10T18:30:47.773 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-10T18:30:47.773 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-a600e180-6fa0-11ef-bcea-c7b262605968@osd.5 2024-09-10T18:30:48.055 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:47 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5[44072]: 2024-09-10T18:30:47.722+0000 7fbd9d5be640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.198315+0000 front 2024-09-10T18:30:09.198366+0000 (oldest deadline 2024-09-10T18:30:34.498606+0000) 2024-09-10T18:30:48.055 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:47 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5[44072]: 2024-09-10T18:30:47.722+0000 7fbd9d5be640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:20.399544+0000 front 2024-09-10T18:30:20.399636+0000 (oldest deadline 2024-09-10T18:30:43.200044+0000) 2024-09-10T18:30:48.055 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:47 smithi150 systemd[1]: Stopping Ceph osd.5 for a600e180-6fa0-11ef-bcea-c7b262605968... 2024-09-10T18:30:48.056 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:47 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:47.760+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.585731+0000 front 2024-09-10T18:30:09.585783+0000 (oldest deadline 2024-09-10T18:30:35.485625+0000) 2024-09-10T18:30:48.056 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:47 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:47.760+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.386848+0000 front 2024-09-10T18:30:21.386832+0000 (oldest deadline 2024-09-10T18:30:44.886674+0000) 2024-09-10T18:30:48.492 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:48 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5[44072]: 2024-09-10T18:30:48.111+0000 7fbda37ab640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:30:48.492 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:48 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5[44072]: 2024-09-10T18:30:48.111+0000 7fbda37ab640 -1 osd.5 66 *** Got signal Terminated *** 2024-09-10T18:30:48.492 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:48 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5[44072]: 2024-09-10T18:30:48.111+0000 7fbda37ab640 -1 osd.5 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:30:48.758 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:48 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:48.489+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:08.860760+0000 front 2024-09-10T18:30:08.860785+0000 (oldest deadline 2024-09-10T18:30:38.261217+0000) 2024-09-10T18:30:48.758 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:48 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:48.489+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.762380+0000 front 2024-09-10T18:30:21.762329+0000 (oldest deadline 2024-09-10T18:30:44.062166+0000) 2024-09-10T18:30:49.055 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:48 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5[44072]: 2024-09-10T18:30:48.756+0000 7fbd9d5be640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.198315+0000 front 2024-09-10T18:30:09.198366+0000 (oldest deadline 2024-09-10T18:30:34.498606+0000) 2024-09-10T18:30:49.055 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:48 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5[44072]: 2024-09-10T18:30:48.756+0000 7fbd9d5be640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:20.399544+0000 front 2024-09-10T18:30:20.399636+0000 (oldest deadline 2024-09-10T18:30:43.200044+0000) 2024-09-10T18:30:49.055 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:48 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:48.760+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.585731+0000 front 2024-09-10T18:30:09.585783+0000 (oldest deadline 2024-09-10T18:30:35.485625+0000) 2024-09-10T18:30:49.055 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:48 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:48.760+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.386848+0000 front 2024-09-10T18:30:21.386832+0000 (oldest deadline 2024-09-10T18:30:44.886674+0000) 2024-09-10T18:30:49.766 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:49 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:49.732+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.585731+0000 front 2024-09-10T18:30:09.585783+0000 (oldest deadline 2024-09-10T18:30:35.485625+0000) 2024-09-10T18:30:49.766 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:49 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:49.732+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.386848+0000 front 2024-09-10T18:30:21.386832+0000 (oldest deadline 2024-09-10T18:30:44.886674+0000) 2024-09-10T18:30:49.766 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:49 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:49.483+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:08.860760+0000 front 2024-09-10T18:30:08.860785+0000 (oldest deadline 2024-09-10T18:30:38.261217+0000) 2024-09-10T18:30:49.766 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:49 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:49.483+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.762380+0000 front 2024-09-10T18:30:21.762329+0000 (oldest deadline 2024-09-10T18:30:44.062166+0000) 2024-09-10T18:30:50.055 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:49 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5[44072]: 2024-09-10T18:30:49.763+0000 7fbd9d5be640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.198315+0000 front 2024-09-10T18:30:09.198366+0000 (oldest deadline 2024-09-10T18:30:34.498606+0000) 2024-09-10T18:30:50.055 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:49 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5[44072]: 2024-09-10T18:30:49.763+0000 7fbd9d5be640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:20.399544+0000 front 2024-09-10T18:30:20.399636+0000 (oldest deadline 2024-09-10T18:30:43.200044+0000) 2024-09-10T18:30:50.736 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:50 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5[44072]: 2024-09-10T18:30:50.734+0000 7fbd9d5be640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.198315+0000 front 2024-09-10T18:30:09.198366+0000 (oldest deadline 2024-09-10T18:30:34.498606+0000) 2024-09-10T18:30:50.737 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:50 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5[44072]: 2024-09-10T18:30:50.734+0000 7fbd9d5be640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:20.399544+0000 front 2024-09-10T18:30:20.399636+0000 (oldest deadline 2024-09-10T18:30:43.200044+0000) 2024-09-10T18:30:50.737 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:50 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:50.476+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:08.860760+0000 front 2024-09-10T18:30:08.860785+0000 (oldest deadline 2024-09-10T18:30:38.261217+0000) 2024-09-10T18:30:50.737 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:50 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:50.476+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.762380+0000 front 2024-09-10T18:30:21.762329+0000 (oldest deadline 2024-09-10T18:30:44.062166+0000) 2024-09-10T18:30:51.054 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:50 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:50.769+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.585731+0000 front 2024-09-10T18:30:09.585783+0000 (oldest deadline 2024-09-10T18:30:35.485625+0000) 2024-09-10T18:30:51.055 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:50 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:50.769+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.386848+0000 front 2024-09-10T18:30:21.386832+0000 (oldest deadline 2024-09-10T18:30:44.886674+0000) 2024-09-10T18:30:51.805 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:51 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5[44072]: 2024-09-10T18:30:51.708+0000 7fbd9d5be640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.198315+0000 front 2024-09-10T18:30:09.198366+0000 (oldest deadline 2024-09-10T18:30:34.498606+0000) 2024-09-10T18:30:51.805 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:51 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5[44072]: 2024-09-10T18:30:51.708+0000 7fbd9d5be640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:20.399544+0000 front 2024-09-10T18:30:20.399636+0000 (oldest deadline 2024-09-10T18:30:43.200044+0000) 2024-09-10T18:30:51.805 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:51 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:51.478+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:08.860760+0000 front 2024-09-10T18:30:08.860785+0000 (oldest deadline 2024-09-10T18:30:38.261217+0000) 2024-09-10T18:30:51.805 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:51 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:51.478+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.762380+0000 front 2024-09-10T18:30:21.762329+0000 (oldest deadline 2024-09-10T18:30:44.062166+0000) 2024-09-10T18:30:52.305 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:51 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:51.805+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.585731+0000 front 2024-09-10T18:30:09.585783+0000 (oldest deadline 2024-09-10T18:30:35.485625+0000) 2024-09-10T18:30:52.305 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:51 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:51.805+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.386848+0000 front 2024-09-10T18:30:21.386832+0000 (oldest deadline 2024-09-10T18:30:44.886674+0000) 2024-09-10T18:30:52.805 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:52 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:52.755+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.585731+0000 front 2024-09-10T18:30:09.585783+0000 (oldest deadline 2024-09-10T18:30:35.485625+0000) 2024-09-10T18:30:52.805 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:52 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:52.755+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.386848+0000 front 2024-09-10T18:30:21.386832+0000 (oldest deadline 2024-09-10T18:30:44.886674+0000) 2024-09-10T18:30:52.805 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:52 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:52.510+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:08.860760+0000 front 2024-09-10T18:30:08.860785+0000 (oldest deadline 2024-09-10T18:30:38.261217+0000) 2024-09-10T18:30:52.805 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:52 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:52.510+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.762380+0000 front 2024-09-10T18:30:21.762329+0000 (oldest deadline 2024-09-10T18:30:44.062166+0000) 2024-09-10T18:30:52.806 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:52 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5[44072]: 2024-09-10T18:30:52.709+0000 7fbd9d5be640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.198315+0000 front 2024-09-10T18:30:09.198366+0000 (oldest deadline 2024-09-10T18:30:34.498606+0000) 2024-09-10T18:30:52.806 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:52 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5[44072]: 2024-09-10T18:30:52.709+0000 7fbd9d5be640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:20.399544+0000 front 2024-09-10T18:30:20.399636+0000 (oldest deadline 2024-09-10T18:30:43.200044+0000) 2024-09-10T18:30:52.806 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:52 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5[44072]: 2024-09-10T18:30:52.709+0000 7fbd9d5be640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T18:30:27.900908+0000 front 2024-09-10T18:30:27.900978+0000 (oldest deadline 2024-09-10T18:30:52.600834+0000) 2024-09-10T18:30:53.779 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:53 smithi150 podman[73661]: 2024-09-10 18:30:53.508214336 +0000 UTC m=+5.513126189 container died c04988644670cd9dd6b786552f5a766f8af79061ef1a9df4c5829006bf110d22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:30:53.779 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:53 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:53.519+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:08.860760+0000 front 2024-09-10T18:30:08.860785+0000 (oldest deadline 2024-09-10T18:30:38.261217+0000) 2024-09-10T18:30:53.779 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:53 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:53.519+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.762380+0000 front 2024-09-10T18:30:21.762329+0000 (oldest deadline 2024-09-10T18:30:44.062166+0000) 2024-09-10T18:30:54.055 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:53 smithi150 podman[73661]: 2024-09-10 18:30:53.892892041 +0000 UTC m=+5.897803894 container cleanup c04988644670cd9dd6b786552f5a766f8af79061ef1a9df4c5829006bf110d22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-10T18:30:54.055 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:53 smithi150 bash[73661]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5 2024-09-10T18:30:54.055 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:53 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:53.777+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.585731+0000 front 2024-09-10T18:30:09.585783+0000 (oldest deadline 2024-09-10T18:30:35.485625+0000) 2024-09-10T18:30:54.055 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:53 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:53.777+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.386848+0000 front 2024-09-10T18:30:21.386832+0000 (oldest deadline 2024-09-10T18:30:44.886674+0000) 2024-09-10T18:30:54.055 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:53 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:53.777+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T18:30:29.487777+0000 front 2024-09-10T18:30:29.487722+0000 (oldest deadline 2024-09-10T18:30:53.587582+0000) 2024-09-10T18:30:54.498 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:54 smithi150 podman[73682]: 2024-09-10 18:30:54.152690241 +0000 UTC m=+0.641395231 container remove c04988644670cd9dd6b786552f5a766f8af79061ef1a9df4c5829006bf110d22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2) 2024-09-10T18:30:54.771 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:54 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:54.495+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:08.860760+0000 front 2024-09-10T18:30:08.860785+0000 (oldest deadline 2024-09-10T18:30:38.261217+0000) 2024-09-10T18:30:54.771 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:54 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:54.495+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.762380+0000 front 2024-09-10T18:30:21.762329+0000 (oldest deadline 2024-09-10T18:30:44.062166+0000) 2024-09-10T18:30:54.772 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:54 smithi150 podman[73774]: 2024-09-10 18:30:54.40355841 +0000 UTC m=+0.015828862 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:30:54.772 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:54 smithi150 podman[73774]: 2024-09-10 18:30:54.520193075 +0000 UTC m=+0.132463511 container create 99844fa1f42c40e837b6072ffef231ff1a1ca39bb67324349204814cb0fdaee7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-10T18:30:54.772 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:54 smithi150 podman[73774]: 2024-09-10 18:30:54.706004667 +0000 UTC m=+0.318275113 container init 99844fa1f42c40e837b6072ffef231ff1a1ca39bb67324349204814cb0fdaee7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-10T18:30:54.772 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:54 smithi150 podman[73774]: 2024-09-10 18:30:54.711171888 +0000 UTC m=+0.323442325 container start 99844fa1f42c40e837b6072ffef231ff1a1ca39bb67324349204814cb0fdaee7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-10T18:30:55.055 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:54 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:54.768+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.585731+0000 front 2024-09-10T18:30:09.585783+0000 (oldest deadline 2024-09-10T18:30:35.485625+0000) 2024-09-10T18:30:55.055 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:54 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:54.768+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.386848+0000 front 2024-09-10T18:30:21.386832+0000 (oldest deadline 2024-09-10T18:30:44.886674+0000) 2024-09-10T18:30:55.055 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:54 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:54.768+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T18:30:29.487777+0000 front 2024-09-10T18:30:29.487722+0000 (oldest deadline 2024-09-10T18:30:53.587582+0000) 2024-09-10T18:30:55.055 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:54 smithi150 podman[73774]: 2024-09-10 18:30:54.805809778 +0000 UTC m=+0.418080223 container attach 99844fa1f42c40e837b6072ffef231ff1a1ca39bb67324349204814cb0fdaee7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=) 2024-09-10T18:30:55.055 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 18:30:54 smithi150 podman[73774]: 2024-09-10 18:30:54.881319526 +0000 UTC m=+0.493589969 container died 99844fa1f42c40e837b6072ffef231ff1a1ca39bb67324349204814cb0fdaee7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-5-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-10T18:30:55.443 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a600e180-6fa0-11ef-bcea-c7b262605968@osd.5.service' 2024-09-10T18:30:55.481 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:30:55.482 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-10T18:30:55.482 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-10T18:30:55.482 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-a600e180-6fa0-11ef-bcea-c7b262605968@osd.6 2024-09-10T18:30:55.729 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:55 smithi150 systemd[1]: Stopping Ceph osd.6 for a600e180-6fa0-11ef-bcea-c7b262605968... 2024-09-10T18:30:55.730 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:55 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:55.512+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:08.860760+0000 front 2024-09-10T18:30:08.860785+0000 (oldest deadline 2024-09-10T18:30:38.261217+0000) 2024-09-10T18:30:55.730 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:55 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:55.512+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.762380+0000 front 2024-09-10T18:30:21.762329+0000 (oldest deadline 2024-09-10T18:30:44.062166+0000) 2024-09-10T18:30:55.730 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:55 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:55.512+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T18:30:29.363256+0000 front 2024-09-10T18:30:29.363141+0000 (oldest deadline 2024-09-10T18:30:54.663001+0000) 2024-09-10T18:30:56.054 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:55 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:55.726+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.585731+0000 front 2024-09-10T18:30:09.585783+0000 (oldest deadline 2024-09-10T18:30:35.485625+0000) 2024-09-10T18:30:56.055 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:55 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:55.726+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.386848+0000 front 2024-09-10T18:30:21.386832+0000 (oldest deadline 2024-09-10T18:30:44.886674+0000) 2024-09-10T18:30:56.055 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:55 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:55.726+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T18:30:29.487777+0000 front 2024-09-10T18:30:29.487722+0000 (oldest deadline 2024-09-10T18:30:53.587582+0000) 2024-09-10T18:30:56.055 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:55 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:55.837+0000 7f017e1d3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:30:56.055 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:55 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:55.837+0000 7f017e1d3640 -1 osd.6 66 *** Got signal Terminated *** 2024-09-10T18:30:56.055 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:55 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:55.837+0000 7f017e1d3640 -1 osd.6 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:30:57.055 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:56.710+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.585731+0000 front 2024-09-10T18:30:09.585783+0000 (oldest deadline 2024-09-10T18:30:35.485625+0000) 2024-09-10T18:30:57.055 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:56.710+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.386848+0000 front 2024-09-10T18:30:21.386832+0000 (oldest deadline 2024-09-10T18:30:44.886674+0000) 2024-09-10T18:30:57.055 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:56.710+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T18:30:29.487777+0000 front 2024-09-10T18:30:29.487722+0000 (oldest deadline 2024-09-10T18:30:53.587582+0000) 2024-09-10T18:30:57.055 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:56.559+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:08.860760+0000 front 2024-09-10T18:30:08.860785+0000 (oldest deadline 2024-09-10T18:30:38.261217+0000) 2024-09-10T18:30:57.056 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:56.559+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.762380+0000 front 2024-09-10T18:30:21.762329+0000 (oldest deadline 2024-09-10T18:30:44.062166+0000) 2024-09-10T18:30:57.056 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:56 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:56.559+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T18:30:29.363256+0000 front 2024-09-10T18:30:29.363141+0000 (oldest deadline 2024-09-10T18:30:54.663001+0000) 2024-09-10T18:30:57.805 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:57 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:57.695+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.585731+0000 front 2024-09-10T18:30:09.585783+0000 (oldest deadline 2024-09-10T18:30:35.485625+0000) 2024-09-10T18:30:57.805 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:57 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:57.695+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.386848+0000 front 2024-09-10T18:30:21.386832+0000 (oldest deadline 2024-09-10T18:30:44.886674+0000) 2024-09-10T18:30:57.805 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:57 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:57.695+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T18:30:29.487777+0000 front 2024-09-10T18:30:29.487722+0000 (oldest deadline 2024-09-10T18:30:53.587582+0000) 2024-09-10T18:30:57.805 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:57 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:57.547+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:08.860760+0000 front 2024-09-10T18:30:08.860785+0000 (oldest deadline 2024-09-10T18:30:38.261217+0000) 2024-09-10T18:30:57.806 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:57 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:57.547+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.762380+0000 front 2024-09-10T18:30:21.762329+0000 (oldest deadline 2024-09-10T18:30:44.062166+0000) 2024-09-10T18:30:57.806 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:57 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:57.547+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T18:30:29.363256+0000 front 2024-09-10T18:30:29.363141+0000 (oldest deadline 2024-09-10T18:30:54.663001+0000) 2024-09-10T18:30:59.055 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:58 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:58.696+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.585731+0000 front 2024-09-10T18:30:09.585783+0000 (oldest deadline 2024-09-10T18:30:35.485625+0000) 2024-09-10T18:30:59.055 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:58 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:58.696+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.386848+0000 front 2024-09-10T18:30:21.386832+0000 (oldest deadline 2024-09-10T18:30:44.886674+0000) 2024-09-10T18:30:59.055 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:58 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:58.696+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T18:30:29.487777+0000 front 2024-09-10T18:30:29.487722+0000 (oldest deadline 2024-09-10T18:30:53.587582+0000) 2024-09-10T18:30:59.055 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:58 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:58.696+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6815 osd.3 since back 2024-09-10T18:30:33.588077+0000 front 2024-09-10T18:30:33.588087+0000 (oldest deadline 2024-09-10T18:30:58.287982+0000) 2024-09-10T18:30:59.056 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:58 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:58.588+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:08.860760+0000 front 2024-09-10T18:30:08.860785+0000 (oldest deadline 2024-09-10T18:30:38.261217+0000) 2024-09-10T18:30:59.056 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:58 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:58.588+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.762380+0000 front 2024-09-10T18:30:21.762329+0000 (oldest deadline 2024-09-10T18:30:44.062166+0000) 2024-09-10T18:30:59.056 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:58 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:58.588+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T18:30:29.363256+0000 front 2024-09-10T18:30:29.363141+0000 (oldest deadline 2024-09-10T18:30:54.663001+0000) 2024-09-10T18:31:00.055 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:59 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:59.673+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.585731+0000 front 2024-09-10T18:30:09.585783+0000 (oldest deadline 2024-09-10T18:30:35.485625+0000) 2024-09-10T18:31:00.055 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:59 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:59.673+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.386848+0000 front 2024-09-10T18:30:21.386832+0000 (oldest deadline 2024-09-10T18:30:44.886674+0000) 2024-09-10T18:31:00.055 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:59 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:59.673+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T18:30:29.487777+0000 front 2024-09-10T18:30:29.487722+0000 (oldest deadline 2024-09-10T18:30:53.587582+0000) 2024-09-10T18:31:00.055 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:30:59 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:30:59.673+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6815 osd.3 since back 2024-09-10T18:30:33.588077+0000 front 2024-09-10T18:30:33.588087+0000 (oldest deadline 2024-09-10T18:30:58.287982+0000) 2024-09-10T18:31:00.055 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:59 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:59.567+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:08.860760+0000 front 2024-09-10T18:30:08.860785+0000 (oldest deadline 2024-09-10T18:30:38.261217+0000) 2024-09-10T18:31:00.055 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:59 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:59.567+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.762380+0000 front 2024-09-10T18:30:21.762329+0000 (oldest deadline 2024-09-10T18:30:44.062166+0000) 2024-09-10T18:31:00.056 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:30:59 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:30:59.567+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T18:30:29.363256+0000 front 2024-09-10T18:30:29.363141+0000 (oldest deadline 2024-09-10T18:30:54.663001+0000) 2024-09-10T18:31:01.055 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:31:00 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:31:00.625+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:09.585731+0000 front 2024-09-10T18:30:09.585783+0000 (oldest deadline 2024-09-10T18:30:35.485625+0000) 2024-09-10T18:31:01.055 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:31:00 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:31:00.625+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.386848+0000 front 2024-09-10T18:30:21.386832+0000 (oldest deadline 2024-09-10T18:30:44.886674+0000) 2024-09-10T18:31:01.055 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:31:00 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:31:00.625+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T18:30:29.487777+0000 front 2024-09-10T18:30:29.487722+0000 (oldest deadline 2024-09-10T18:30:53.587582+0000) 2024-09-10T18:31:01.055 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:31:00 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6[50428]: 2024-09-10T18:31:00.625+0000 7f01797e9640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.115:6815 osd.3 since back 2024-09-10T18:30:33.588077+0000 front 2024-09-10T18:30:33.588087+0000 (oldest deadline 2024-09-10T18:30:58.287982+0000) 2024-09-10T18:31:01.055 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:00 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:31:00.586+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:08.860760+0000 front 2024-09-10T18:30:08.860785+0000 (oldest deadline 2024-09-10T18:30:38.261217+0000) 2024-09-10T18:31:01.055 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:00 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:31:00.586+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.762380+0000 front 2024-09-10T18:30:21.762329+0000 (oldest deadline 2024-09-10T18:30:44.062166+0000) 2024-09-10T18:31:01.055 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:00 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:31:00.586+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T18:30:29.363256+0000 front 2024-09-10T18:30:29.363141+0000 (oldest deadline 2024-09-10T18:30:54.663001+0000) 2024-09-10T18:31:01.056 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:00 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:31:00.586+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6815 osd.3 since back 2024-09-10T18:30:35.763938+0000 front 2024-09-10T18:30:35.763953+0000 (oldest deadline 2024-09-10T18:30:59.863820+0000) 2024-09-10T18:31:01.555 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:31:01 smithi150 podman[73900]: 2024-09-10 18:31:01.29673594 +0000 UTC m=+5.595193837 container died f8936e76b2eaa76d9c275b1e10b5152a15a32f6db2073f0f906bc09dfcfd3350 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:31:01.947 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:31:01 smithi150 podman[73900]: 2024-09-10 18:31:01.703674111 +0000 UTC m=+6.002132005 container cleanup f8936e76b2eaa76d9c275b1e10b5152a15a32f6db2073f0f906bc09dfcfd3350 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2) 2024-09-10T18:31:01.948 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:31:01 smithi150 bash[73900]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6 2024-09-10T18:31:01.948 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:01 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:31:01.610+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:08.860760+0000 front 2024-09-10T18:30:08.860785+0000 (oldest deadline 2024-09-10T18:30:38.261217+0000) 2024-09-10T18:31:01.948 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:01 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:31:01.610+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.762380+0000 front 2024-09-10T18:30:21.762329+0000 (oldest deadline 2024-09-10T18:30:44.062166+0000) 2024-09-10T18:31:01.948 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:01 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:31:01.610+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T18:30:29.363256+0000 front 2024-09-10T18:30:29.363141+0000 (oldest deadline 2024-09-10T18:30:54.663001+0000) 2024-09-10T18:31:01.948 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:01 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:31:01.610+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6815 osd.3 since back 2024-09-10T18:30:35.763938+0000 front 2024-09-10T18:30:35.763953+0000 (oldest deadline 2024-09-10T18:30:59.863820+0000) 2024-09-10T18:31:02.256 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:31:01 smithi150 podman[73922]: 2024-09-10 18:31:01.946587403 +0000 UTC m=+0.642719173 container remove f8936e76b2eaa76d9c275b1e10b5152a15a32f6db2073f0f906bc09dfcfd3350 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:31:02.542 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:31:02 smithi150 podman[74020]: 2024-09-10 18:31:02.158041733 +0000 UTC m=+0.016095471 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:31:02.542 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:31:02 smithi150 podman[74020]: 2024-09-10 18:31:02.264008832 +0000 UTC m=+0.122062607 container create 7053a095d7e0ee7de32d40ef61e3a3c80834753a1043c0df85adbe72a7c9cc26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-10T18:31:02.542 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:31:02 smithi150 podman[74020]: 2024-09-10 18:31:02.449801094 +0000 UTC m=+0.307854826 container init 7053a095d7e0ee7de32d40ef61e3a3c80834753a1043c0df85adbe72a7c9cc26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731) 2024-09-10T18:31:02.542 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:31:02 smithi150 podman[74020]: 2024-09-10 18:31:02.454352995 +0000 UTC m=+0.312406728 container start 7053a095d7e0ee7de32d40ef61e3a3c80834753a1043c0df85adbe72a7c9cc26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True) 2024-09-10T18:31:02.542 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:31:02 smithi150 podman[74020]: 2024-09-10 18:31:02.541033686 +0000 UTC m=+0.399087429 container attach 7053a095d7e0ee7de32d40ef61e3a3c80834753a1043c0df85adbe72a7c9cc26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T18:31:02.805 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:31:02 smithi150 podman[74020]: 2024-09-10 18:31:02.628736233 +0000 UTC m=+0.486789966 container died 7053a095d7e0ee7de32d40ef61e3a3c80834753a1043c0df85adbe72a7c9cc26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-10T18:31:02.805 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:02 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:31:02.562+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:08.860760+0000 front 2024-09-10T18:30:08.860785+0000 (oldest deadline 2024-09-10T18:30:38.261217+0000) 2024-09-10T18:31:02.805 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:02 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:31:02.562+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.762380+0000 front 2024-09-10T18:30:21.762329+0000 (oldest deadline 2024-09-10T18:30:44.062166+0000) 2024-09-10T18:31:02.805 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:02 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:31:02.562+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T18:30:29.363256+0000 front 2024-09-10T18:30:29.363141+0000 (oldest deadline 2024-09-10T18:30:54.663001+0000) 2024-09-10T18:31:02.805 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:02 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:31:02.562+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6815 osd.3 since back 2024-09-10T18:30:35.763938+0000 front 2024-09-10T18:30:35.763953+0000 (oldest deadline 2024-09-10T18:30:59.863820+0000) 2024-09-10T18:31:03.136 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a600e180-6fa0-11ef-bcea-c7b262605968@osd.6.service' 2024-09-10T18:31:03.159 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:31:03 smithi150 podman[74063]: 2024-09-10 18:31:03.119456072 +0000 UTC m=+0.485265931 container remove 7053a095d7e0ee7de32d40ef61e3a3c80834753a1043c0df85adbe72a7c9cc26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731) 2024-09-10T18:31:03.159 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:31:03 smithi150 systemd[1]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968@osd.6.service: Deactivated successfully. 2024-09-10T18:31:03.159 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:31:03 smithi150 systemd[1]: Stopped Ceph osd.6 for a600e180-6fa0-11ef-bcea-c7b262605968. 2024-09-10T18:31:03.159 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 18:31:03 smithi150 systemd[1]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968@osd.6.service: Consumed 4.585s CPU time. 2024-09-10T18:31:03.174 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:31:03.174 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-10T18:31:03.174 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-10T18:31:03.174 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-a600e180-6fa0-11ef-bcea-c7b262605968@osd.7 2024-09-10T18:31:03.509 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:03 smithi150 systemd[1]: Stopping Ceph osd.7 for a600e180-6fa0-11ef-bcea-c7b262605968... 2024-09-10T18:31:03.805 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:03 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:31:03.507+0000 7f7f04b80640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:31:03.805 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:03 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:31:03.507+0000 7f7f04b80640 -1 osd.7 66 *** Got signal Terminated *** 2024-09-10T18:31:03.805 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:03 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:31:03.507+0000 7f7f04b80640 -1 osd.7 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:31:03.805 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:03 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:31:03.575+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:08.860760+0000 front 2024-09-10T18:30:08.860785+0000 (oldest deadline 2024-09-10T18:30:38.261217+0000) 2024-09-10T18:31:03.805 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:03 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:31:03.575+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.762380+0000 front 2024-09-10T18:30:21.762329+0000 (oldest deadline 2024-09-10T18:30:44.062166+0000) 2024-09-10T18:31:03.805 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:03 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:31:03.575+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T18:30:29.363256+0000 front 2024-09-10T18:30:29.363141+0000 (oldest deadline 2024-09-10T18:30:54.663001+0000) 2024-09-10T18:31:03.806 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:03 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:31:03.575+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6815 osd.3 since back 2024-09-10T18:30:35.763938+0000 front 2024-09-10T18:30:35.763953+0000 (oldest deadline 2024-09-10T18:30:59.863820+0000) 2024-09-10T18:31:05.055 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:04 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:31:04.597+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:08.860760+0000 front 2024-09-10T18:30:08.860785+0000 (oldest deadline 2024-09-10T18:30:38.261217+0000) 2024-09-10T18:31:05.055 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:04 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:31:04.597+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.762380+0000 front 2024-09-10T18:30:21.762329+0000 (oldest deadline 2024-09-10T18:30:44.062166+0000) 2024-09-10T18:31:05.055 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:04 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:31:04.597+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T18:30:29.363256+0000 front 2024-09-10T18:30:29.363141+0000 (oldest deadline 2024-09-10T18:30:54.663001+0000) 2024-09-10T18:31:05.055 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:04 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:31:04.597+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6815 osd.3 since back 2024-09-10T18:30:35.763938+0000 front 2024-09-10T18:30:35.763953+0000 (oldest deadline 2024-09-10T18:30:59.863820+0000) 2024-09-10T18:31:05.956 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:05 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:31:05.583+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:08.860760+0000 front 2024-09-10T18:30:08.860785+0000 (oldest deadline 2024-09-10T18:30:38.261217+0000) 2024-09-10T18:31:05.956 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:05 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:31:05.583+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.762380+0000 front 2024-09-10T18:30:21.762329+0000 (oldest deadline 2024-09-10T18:30:44.062166+0000) 2024-09-10T18:31:05.956 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:05 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:31:05.583+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T18:30:29.363256+0000 front 2024-09-10T18:30:29.363141+0000 (oldest deadline 2024-09-10T18:30:54.663001+0000) 2024-09-10T18:31:05.956 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:05 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:31:05.583+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6815 osd.3 since back 2024-09-10T18:30:35.763938+0000 front 2024-09-10T18:30:35.763953+0000 (oldest deadline 2024-09-10T18:30:59.863820+0000) 2024-09-10T18:31:07.055 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:06 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:31:06.605+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:08.860760+0000 front 2024-09-10T18:30:08.860785+0000 (oldest deadline 2024-09-10T18:30:38.261217+0000) 2024-09-10T18:31:07.055 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:06 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:31:06.605+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.762380+0000 front 2024-09-10T18:30:21.762329+0000 (oldest deadline 2024-09-10T18:30:44.062166+0000) 2024-09-10T18:31:07.055 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:06 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:31:06.605+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T18:30:29.363256+0000 front 2024-09-10T18:30:29.363141+0000 (oldest deadline 2024-09-10T18:30:54.663001+0000) 2024-09-10T18:31:07.055 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:06 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:31:06.605+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6815 osd.3 since back 2024-09-10T18:30:35.763938+0000 front 2024-09-10T18:30:35.763953+0000 (oldest deadline 2024-09-10T18:30:59.863820+0000) 2024-09-10T18:31:08.055 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:07 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:31:07.636+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T18:30:08.860760+0000 front 2024-09-10T18:30:08.860785+0000 (oldest deadline 2024-09-10T18:30:38.261217+0000) 2024-09-10T18:31:08.055 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:07 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:31:07.636+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T18:30:21.762380+0000 front 2024-09-10T18:30:21.762329+0000 (oldest deadline 2024-09-10T18:30:44.062166+0000) 2024-09-10T18:31:08.055 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:07 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:31:07.636+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T18:30:29.363256+0000 front 2024-09-10T18:30:29.363141+0000 (oldest deadline 2024-09-10T18:30:54.663001+0000) 2024-09-10T18:31:08.055 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:07 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7[56657]: 2024-09-10T18:31:07.636+0000 7f7f0299b640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.115:6815 osd.3 since back 2024-09-10T18:30:35.763938+0000 front 2024-09-10T18:30:35.763953+0000 (oldest deadline 2024-09-10T18:30:59.863820+0000) 2024-09-10T18:31:09.305 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:09 smithi150 podman[74148]: 2024-09-10 18:31:09.017730778 +0000 UTC m=+5.616356540 container died 45afcee829060a0205bdeab2281125a1c0ceb745ef8e208ad780602f79882fb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-10T18:31:09.657 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:09 smithi150 podman[74148]: 2024-09-10 18:31:09.405354404 +0000 UTC m=+6.003980204 container cleanup 45afcee829060a0205bdeab2281125a1c0ceb745ef8e208ad780602f79882fb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:31:09.657 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:09 smithi150 bash[74148]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7 2024-09-10T18:31:09.966 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:09 smithi150 podman[74183]: 2024-09-10 18:31:09.656278168 +0000 UTC m=+0.633579209 container remove 45afcee829060a0205bdeab2281125a1c0ceb745ef8e208ad780602f79882fb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-10T18:31:10.227 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:09 smithi150 podman[74277]: 2024-09-10 18:31:09.965343283 +0000 UTC m=+0.104937540 container create ce33280d5e1899864e0311cdc07c8251b3c1b5ba454cfeb9da217e771492a5aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7-deactivate, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-10T18:31:10.227 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:09 smithi150 podman[74277]: 2024-09-10 18:31:09.875786938 +0000 UTC m=+0.015381195 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:31:10.227 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:10 smithi150 podman[74277]: 2024-09-10 18:31:10.150860897 +0000 UTC m=+0.290455169 container init ce33280d5e1899864e0311cdc07c8251b3c1b5ba454cfeb9da217e771492a5aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD) 2024-09-10T18:31:10.227 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:10 smithi150 podman[74277]: 2024-09-10 18:31:10.155688162 +0000 UTC m=+0.295282419 container start ce33280d5e1899864e0311cdc07c8251b3c1b5ba454cfeb9da217e771492a5aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7-deactivate, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux ) 2024-09-10T18:31:10.227 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:10 smithi150 podman[74277]: 2024-09-10 18:31:10.225653196 +0000 UTC m=+0.365247474 container attach ce33280d5e1899864e0311cdc07c8251b3c1b5ba454cfeb9da217e771492a5aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:31:10.555 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:10 smithi150 podman[74277]: 2024-09-10 18:31:10.333317677 +0000 UTC m=+0.472911938 container died ce33280d5e1899864e0311cdc07c8251b3c1b5ba454cfeb9da217e771492a5aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7-deactivate, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-10T18:31:10.890 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a600e180-6fa0-11ef-bcea-c7b262605968@osd.7.service' 2024-09-10T18:31:10.913 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:10 smithi150 podman[74321]: 2024-09-10 18:31:10.86225334 +0000 UTC m=+0.522849910 container remove ce33280d5e1899864e0311cdc07c8251b3c1b5ba454cfeb9da217e771492a5aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:31:10.914 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:10 smithi150 systemd[1]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968@osd.7.service: Deactivated successfully. 2024-09-10T18:31:10.914 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:10 smithi150 systemd[1]: Stopped Ceph osd.7 for a600e180-6fa0-11ef-bcea-c7b262605968. 2024-09-10T18:31:10.914 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 18:31:10 smithi150 systemd[1]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968@osd.7.service: Consumed 4.935s CPU time. 2024-09-10T18:31:10.929 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:31:10.929 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-10T18:31:10.929 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-09-10T18:31:10.929 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-a600e180-6fa0-11ef-bcea-c7b262605968@rgw.foo.a 2024-09-10T18:31:11.234 INFO:journalctl@ceph.rgw.foo.a.smithi115.stdout:Sep 10 18:31:10 smithi115 systemd[1]: Stopping Ceph rgw.foo.a for a600e180-6fa0-11ef-bcea-c7b262605968... 2024-09-10T18:31:11.488 INFO:journalctl@ceph.rgw.foo.a.smithi115.stdout:Sep 10 18:31:11 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-rgw-foo-a[69404]: 2024-09-10T18:31:11.301+0000 7f6a228fc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:31:11.488 INFO:journalctl@ceph.rgw.foo.a.smithi115.stdout:Sep 10 18:31:11 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-rgw-foo-a[69404]: 2024-09-10T18:31:11.301+0000 7f6a266037c0 -1 shutting down 2024-09-10T18:31:11.985 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:31:11 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a[79329]: ts=2024-09-10T18:31:11.628Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi115.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.115:8443: connect: connection refused" 2024-09-10T18:31:11.985 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:31:11 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a[79329]: ts=2024-09-10T18:31:11.629Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi150.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.150:8443: connect: connection refused" 2024-09-10T18:31:21.630 INFO:journalctl@ceph.rgw.foo.a.smithi115.stdout:Sep 10 18:31:21 smithi115 bash[88734]: time="2024-09-10T18:31:21Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-a600e180-6fa0-11ef-bcea-c7b262605968-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-09-10T18:31:21.631 INFO:journalctl@ceph.rgw.foo.a.smithi115.stdout:Sep 10 18:31:21 smithi115 podman[88734]: 2024-09-10 18:31:21.33147802 +0000 UTC m=+10.208768878 container died 31854f3a8e9ff3fda99bceace9bf21c1b547badb37df39130908571a36c3ebd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-rgw-foo-a, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-10T18:31:21.885 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:31:21 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a[79329]: ts=2024-09-10T18:31:21.628Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi150.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.150:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi115.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.115:8443: connect: connection refused" 2024-09-10T18:31:21.885 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:31:21 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a[79329]: ts=2024-09-10T18:31:21.629Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi115.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.115:8443: connect: connection refused" 2024-09-10T18:31:21.885 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:31:21 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a[79329]: ts=2024-09-10T18:31:21.630Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi150.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.150:8443: connect: connection refused" 2024-09-10T18:31:22.159 INFO:journalctl@ceph.rgw.foo.a.smithi115.stdout:Sep 10 18:31:21 smithi115 podman[88734]: 2024-09-10 18:31:21.973558643 +0000 UTC m=+10.850849509 container cleanup 31854f3a8e9ff3fda99bceace9bf21c1b547badb37df39130908571a36c3ebd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-rgw-foo-a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2) 2024-09-10T18:31:22.159 INFO:journalctl@ceph.rgw.foo.a.smithi115.stdout:Sep 10 18:31:21 smithi115 bash[88734]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968-rgw-foo-a 2024-09-10T18:31:22.394 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a600e180-6fa0-11ef-bcea-c7b262605968@rgw.foo.a.service' 2024-09-10T18:31:22.417 INFO:journalctl@ceph.rgw.foo.a.smithi115.stdout:Sep 10 18:31:22 smithi115 podman[88770]: 2024-09-10 18:31:22.157947163 +0000 UTC m=+0.822242306 container remove 31854f3a8e9ff3fda99bceace9bf21c1b547badb37df39130908571a36c3ebd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-rgw-foo-a, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-10T18:31:22.417 INFO:journalctl@ceph.rgw.foo.a.smithi115.stdout:Sep 10 18:31:22 smithi115 systemd[1]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968@rgw.foo.a.service: Main process exited, code=exited, status=137/n/a 2024-09-10T18:31:22.417 INFO:journalctl@ceph.rgw.foo.a.smithi115.stdout:Sep 10 18:31:22 smithi115 systemd[1]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968@rgw.foo.a.service: Failed with result 'exit-code'. 2024-09-10T18:31:22.417 INFO:journalctl@ceph.rgw.foo.a.smithi115.stdout:Sep 10 18:31:22 smithi115 systemd[1]: Stopped Ceph rgw.foo.a for a600e180-6fa0-11ef-bcea-c7b262605968. 2024-09-10T18:31:22.417 INFO:journalctl@ceph.rgw.foo.a.smithi115.stdout:Sep 10 18:31:22 smithi115 systemd[1]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968@rgw.foo.a.service: Consumed 2.149s CPU time. 2024-09-10T18:31:22.430 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:31:22.430 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-09-10T18:31:22.430 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-10T18:31:22.431 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-a600e180-6fa0-11ef-bcea-c7b262605968@prometheus.a 2024-09-10T18:31:22.743 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:31:22 smithi150 systemd[1]: Stopping Ceph prometheus.a for a600e180-6fa0-11ef-bcea-c7b262605968... 2024-09-10T18:31:23.048 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:31:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[67544]: ts=2024-09-10T18:31:22.741Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-10T18:31:23.048 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:31:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[67544]: ts=2024-09-10T18:31:22.742Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-10T18:31:23.048 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:31:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[67544]: ts=2024-09-10T18:31:22.742Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-10T18:31:23.048 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:31:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[67544]: ts=2024-09-10T18:31:22.742Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-10T18:31:23.049 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:31:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[67544]: ts=2024-09-10T18:31:22.742Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-10T18:31:23.049 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:31:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[67544]: ts=2024-09-10T18:31:22.742Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-10T18:31:23.049 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:31:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[67544]: ts=2024-09-10T18:31:22.742Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-10T18:31:23.049 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:31:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[67544]: ts=2024-09-10T18:31:22.742Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-10T18:31:23.049 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:31:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[67544]: ts=2024-09-10T18:31:22.742Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-10T18:31:23.049 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:31:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[67544]: ts=2024-09-10T18:31:22.742Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-10T18:31:23.049 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:31:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[67544]: ts=2024-09-10T18:31:22.742Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-10T18:31:23.049 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:31:22 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a[67544]: ts=2024-09-10T18:31:22.742Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-10T18:31:23.049 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:31:22 smithi150 podman[74406]: 2024-09-10 18:31:22.752584639 +0000 UTC m=+0.128279462 container died c79b961f5b3989fd58dd376cec399bc4747a662e805336e650dd7969e8a923a4 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:31:23.049 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:31:23 smithi150 podman[74406]: 2024-09-10 18:31:23.047069799 +0000 UTC m=+0.422764621 container cleanup c79b961f5b3989fd58dd376cec399bc4747a662e805336e650dd7969e8a923a4 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:31:23.049 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:31:23 smithi150 bash[74406]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a 2024-09-10T18:31:23.304 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 18:31:23 smithi150 podman[74424]: 2024-09-10 18:31:23.281615488 +0000 UTC m=+0.531232041 container remove c79b961f5b3989fd58dd376cec399bc4747a662e805336e650dd7969e8a923a4 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:31:23.418 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a600e180-6fa0-11ef-bcea-c7b262605968@prometheus.a.service' 2024-09-10T18:31:23.452 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:31:23.453 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-10T18:31:23.453 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm rm-cluster --fsid a600e180-6fa0-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T18:31:25.235 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:31:25 smithi115 systemd[1]: Stopping Ceph alertmanager.a for a600e180-6fa0-11ef-bcea-c7b262605968... 2024-09-10T18:31:25.672 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:31:25 smithi115 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a[79329]: ts=2024-09-10T18:31:25.381Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-10T18:31:25.672 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:31:25 smithi115 podman[89206]: 2024-09-10 18:31:25.401641918 +0000 UTC m=+0.164934277 container died 304ebc95de84e82ae3c5e7ae709d924a70669a072df16b06f1cbd673e863f728 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:31:25.984 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:31:25 smithi115 podman[89206]: 2024-09-10 18:31:25.770043312 +0000 UTC m=+0.533335680 container cleanup 304ebc95de84e82ae3c5e7ae709d924a70669a072df16b06f1cbd673e863f728 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:31:25.985 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:31:25 smithi115 bash[89206]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a 2024-09-10T18:31:26.326 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:31:26 smithi115 podman[89226]: 2024-09-10 18:31:26.037436333 +0000 UTC m=+0.637770371 container remove 304ebc95de84e82ae3c5e7ae709d924a70669a072df16b06f1cbd673e863f728 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:31:26.327 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:31:26 smithi115 podman[89226]: 2024-09-10 18:31:26.12132761 +0000 UTC m=+0.721661646 volume remove c0bde6ca00d4ff184b14ccd00780baa487a8e5351f471fd567f14e2827333b12 2024-09-10T18:31:26.735 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:31:26 smithi115 systemd[1]: Stopping Ceph node-exporter.a for a600e180-6fa0-11ef-bcea-c7b262605968... 2024-09-10T18:31:26.735 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:31:26 smithi115 systemd[1]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-10T18:31:26.735 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:31:26 smithi115 systemd[1]: Stopped Ceph alertmanager.a for a600e180-6fa0-11ef-bcea-c7b262605968. 2024-09-10T18:31:26.735 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 18:31:26 smithi115 systemd[1]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968@alertmanager.a.service: Consumed 1.231s CPU time. 2024-09-10T18:31:27.042 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:31:26 smithi115 podman[89336]: 2024-09-10 18:31:26.772872 +0000 UTC m=+0.124452078 container died 5269993361eec018059e5fcdaef113dbeb3f249f4eea9a4fb8d4f543b025a36e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T18:31:27.417 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:31:27 smithi115 podman[89336]: 2024-09-10 18:31:27.132155753 +0000 UTC m=+0.483735831 container cleanup 5269993361eec018059e5fcdaef113dbeb3f249f4eea9a4fb8d4f543b025a36e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T18:31:27.418 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:31:27 smithi115 bash[89336]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a 2024-09-10T18:31:27.688 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:31:27 smithi115 podman[89355]: 2024-09-10 18:31:27.416612939 +0000 UTC m=+0.647748396 container remove 5269993361eec018059e5fcdaef113dbeb3f249f4eea9a4fb8d4f543b025a36e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T18:31:27.689 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:31:27 smithi115 systemd[1]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-10T18:31:27.689 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:31:27 smithi115 systemd[1]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-10T18:31:27.689 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:31:27 smithi115 systemd[1]: Stopped Ceph node-exporter.a for a600e180-6fa0-11ef-bcea-c7b262605968. 2024-09-10T18:31:27.689 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 18:31:27 smithi115 systemd[1]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968@node-exporter.a.service: Consumed 2.501s CPU time. 2024-09-10T18:31:28.160 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm rm-cluster --fsid a600e180-6fa0-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T18:31:29.748 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:31:29 smithi150 systemd[1]: Stopping Ceph iscsi.iscsi.a for a600e180-6fa0-11ef-bcea-c7b262605968... 2024-09-10T18:31:30.054 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:31:29 smithi150 ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a[62196]: debug Shutdown received 2024-09-10T18:31:40.011 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:31:39 smithi150 bash[74787]: time="2024-09-10T18:31:39Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-09-10T18:31:40.011 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:31:39 smithi150 podman[74787]: 2024-09-10 18:31:39.772785801 +0000 UTC m=+10.129318504 container died 8aed98e74f71fceaa186e06c34dc1b969eccbff5c2f348c12af84787e9c2cb20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T18:31:40.304 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:31:40 smithi150 podman[74787]: 2024-09-10 18:31:40.117665626 +0000 UTC m=+10.474198332 container cleanup 8aed98e74f71fceaa186e06c34dc1b969eccbff5c2f348c12af84787e9c2cb20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD) 2024-09-10T18:31:40.305 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:31:40 smithi150 bash[74787]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a 2024-09-10T18:31:40.805 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:31:40 smithi150 podman[74811]: 2024-09-10 18:31:40.418835591 +0000 UTC m=+0.646662972 container remove 8aed98e74f71fceaa186e06c34dc1b969eccbff5c2f348c12af84787e9c2cb20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-10T18:31:40.805 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:31:40 smithi150 systemd[1]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-09-10T18:31:40.805 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:31:40 smithi150 podman[74858]: 2024-09-10 18:31:40.650136222 +0000 UTC m=+0.113238524 container died c03d5be2b109152142e55d2bc9822e052602b3a7e998b2dd12edecfd15ac2ffc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-10T18:31:41.234 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:31:41 smithi150 podman[74858]: 2024-09-10 18:31:41.004237647 +0000 UTC m=+0.467339939 container cleanup c03d5be2b109152142e55d2bc9822e052602b3a7e998b2dd12edecfd15ac2ffc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-10T18:31:41.234 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:31:41 smithi150 bash[74858]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu 2024-09-10T18:31:41.234 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:31:41 smithi150 systemd[1]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-09-10T18:31:41.234 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:31:41 smithi150 systemd[1]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Unit process 62027 (conmon) remains running after unit stopped. 2024-09-10T18:31:41.235 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:31:41 smithi150 systemd[1]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Unit process 74875 (podman) remains running after unit stopped. 2024-09-10T18:31:41.235 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:31:41 smithi150 systemd[1]: Stopped Ceph iscsi.iscsi.a for a600e180-6fa0-11ef-bcea-c7b262605968. 2024-09-10T18:31:41.235 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:31:41 smithi150 systemd[1]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Consumed 2.668s CPU time. 2024-09-10T18:31:41.515 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:31:41 smithi150 systemd[1]: Stopping Ceph grafana.a for a600e180-6fa0-11ef-bcea-c7b262605968... 2024-09-10T18:31:41.516 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 18:31:41 smithi150 podman[74875]: 2024-09-10 18:31:41.314359423 +0000 UTC m=+0.666325172 container remove c03d5be2b109152142e55d2bc9822e052602b3a7e998b2dd12edecfd15ac2ffc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-10T18:31:41.804 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:31:41 smithi150 podman[75005]: 2024-09-10 18:31:41.67192585 +0000 UTC m=+0.087386742 container died 92ac408df0e8fa73388979915d6a351b986fe781a6e6bb9f3595aa055ac9368d (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a, com.redhat.component=ubi8-container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.k8s.display-name=Red Hat Universal Base Image 8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.openshift.expose-services=, io.openshift.tags=base rhel8, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, version=8.5, io.buildah.version=1.32.0, vcs-type=git, distribution-scope=public, summary=Grafana Container configured for Ceph mgr/dashboard integration, build-date=2022-04-29T11:48:11.055897, vendor=Red Hat, Inc., description=Ceph Grafana Container, name=ubi8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, maintainer=Paul Cuzner , architecture=x86_64, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, release=239.1651231664) 2024-09-10T18:31:42.301 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:31:41 smithi150 podman[75005]: 2024-09-10 18:31:41.983460395 +0000 UTC m=+0.398921285 container cleanup 92ac408df0e8fa73388979915d6a351b986fe781a6e6bb9f3595aa055ac9368d (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a, architecture=x86_64, summary=Grafana Container configured for Ceph mgr/dashboard integration, build-date=2022-04-29T11:48:11.055897, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.buildah.version=1.32.0, name=ubi8, io.openshift.expose-services=, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., description=Ceph Grafana Container, version=8.5, vcs-type=git, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.component=ubi8-container, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, maintainer=Paul Cuzner , release=239.1651231664, vendor=Red Hat, Inc., distribution-scope=public, io.openshift.tags=base rhel8) 2024-09-10T18:31:42.301 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:31:41 smithi150 bash[75005]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a 2024-09-10T18:31:42.301 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:31:42 smithi150 podman[75022]: 2024-09-10 18:31:42.142938814 +0000 UTC m=+0.473440805 container remove 92ac408df0e8fa73388979915d6a351b986fe781a6e6bb9f3595aa055ac9368d (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-grafana-a, com.redhat.component=ubi8-container, io.buildah.version=1.32.0, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, release=239.1651231664, vendor=Red Hat, Inc., description=Ceph Grafana Container, distribution-scope=public, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, version=8.5, architecture=x86_64, build-date=2022-04-29T11:48:11.055897, io.openshift.expose-services=, maintainer=Paul Cuzner , url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vcs-type=git, summary=Grafana Container configured for Ceph mgr/dashboard integration, name=ubi8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.tags=base rhel8) 2024-09-10T18:31:42.301 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:31:42 smithi150 systemd[1]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-09-10T18:31:42.557 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:31:42 smithi150 systemd[1]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968@grafana.a.service: Failed with result 'exit-code'. 2024-09-10T18:31:42.557 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:31:42 smithi150 systemd[1]: Stopped Ceph grafana.a for a600e180-6fa0-11ef-bcea-c7b262605968. 2024-09-10T18:31:42.557 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 18:31:42 smithi150 systemd[1]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968@grafana.a.service: Consumed 6.858s CPU time. 2024-09-10T18:31:43.014 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:31:42 smithi150 systemd[1]: Stopping Ceph node-exporter.b for a600e180-6fa0-11ef-bcea-c7b262605968... 2024-09-10T18:31:43.014 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:31:42 smithi150 podman[75136]: 2024-09-10 18:31:42.800426472 +0000 UTC m=+0.098053718 container died 95530b4a3c723a92014cba595895037f7ac0a800bb7fd3569a7c62769b225c9a (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T18:31:43.305 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:31:43 smithi150 podman[75136]: 2024-09-10 18:31:43.12068831 +0000 UTC m=+0.418315556 container cleanup 95530b4a3c723a92014cba595895037f7ac0a800bb7fd3569a7c62769b225c9a (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T18:31:43.305 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:31:43 smithi150 bash[75136]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b 2024-09-10T18:31:43.672 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:31:43 smithi150 podman[75155]: 2024-09-10 18:31:43.305132794 +0000 UTC m=+0.504977689 container remove 95530b4a3c723a92014cba595895037f7ac0a800bb7fd3569a7c62769b225c9a (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-a600e180-6fa0-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T18:31:43.672 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:31:43 smithi150 systemd[1]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-10T18:31:43.672 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:31:43 smithi150 systemd[1]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-10T18:31:43.672 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:31:43 smithi150 systemd[1]: Stopped Ceph node-exporter.b for a600e180-6fa0-11ef-bcea-c7b262605968. 2024-09-10T18:31:43.672 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 18:31:43 smithi150 systemd[1]: ceph-a600e180-6fa0-11ef-bcea-c7b262605968@node-exporter.b.service: Consumed 2.519s CPU time. 2024-09-10T18:31:44.183 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:31:44.216 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:31:44.244 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-10T18:31:44.246 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900210/remote/smithi115/crash 2024-09-10T18:31:44.247 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/crash -- . 2024-09-10T18:31:44.326 INFO:teuthology.orchestra.run.smithi115.stderr:tar: /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T18:31:44.326 INFO:teuthology.orchestra.run.smithi115.stderr:tar: Error is not recoverable: exiting now 2024-09-10T18:31:44.328 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900210/remote/smithi150/crash 2024-09-10T18:31:44.329 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/crash -- . 2024-09-10T18:31:44.365 INFO:teuthology.orchestra.run.smithi150.stderr:tar: /var/lib/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T18:31:44.365 INFO:teuthology.orchestra.run.smithi150.stderr:tar: Error is not recoverable: exiting now 2024-09-10T18:31:44.366 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-10T18:31:44.366 DEBUG:teuthology.orchestra.run.smithi115:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-09-10T18:31:44.409 INFO:teuthology.orchestra.run.smithi115.stderr:grep: /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph.log: No such file or directory 2024-09-10T18:31:44.411 INFO:tasks.cephadm:Compressing logs... 2024-09-10T18:31:44.411 DEBUG:teuthology.orchestra.run.smithi115:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T18:31:44.454 DEBUG:teuthology.orchestra.run.smithi150:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T18:31:44.481 INFO:teuthology.orchestra.run.smithi150.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/tcmu-runner.log 2024-09-10T18:31:44.481 INFO:teuthology.orchestra.run.smithi150.stderr:: No such file or directory 2024-09-10T18:31:44.481 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-10T18:31:44.481 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/tcmu-runner.log: gzip -5 --verbose -- /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T18:31:44.481 INFO:teuthology.orchestra.run.smithi150.stderr: 63.7% -- replaced with /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/tcmu-runner.log.gz 2024-09-10T18:31:44.481 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-10T18:31:44.482 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-10T18:31:44.482 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-10T18:31:44.482 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-10T18:31:44.482 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-10T18:31:44.483 INFO:teuthology.orchestra.run.smithi115.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-10T18:31:44.483 INFO:teuthology.orchestra.run.smithi115.stderr:: No such file or directory 2024-09-10T18:31:44.484 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-10T18:31:44.484 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T18:31:44.484 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-10T18:31:44.484 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-10T18:31:44.485 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T18:31:44.485 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-mon.b.log: /var/log/ceph/cephadm.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T18:31:44.485 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-volume.log: /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-10T18:31:44.487 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-10T18:31:44.487 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-osd.2.log: /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log 2024-09-10T18:31:44.491 INFO:teuthology.orchestra.run.smithi150.stderr: 91.2% -- replaced with /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T18:31:44.495 INFO:teuthology.orchestra.run.smithi115.stderr: 91.2% -- replaced with /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T18:31:44.495 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-10T18:31:44.496 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log: 58.7% -- replaced with /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log.gz 2024-09-10T18:31:44.497 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T18:31:44.507 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-10T18:31:44.513 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 91.7% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T18:31:44.700 INFO:teuthology.orchestra.run.smithi115.stderr: 88.9% -- replaced with /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-10T18:31:44.720 INFO:teuthology.orchestra.run.smithi150.stderr: 92.2% -- replaced with /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-10T18:31:44.771 INFO:teuthology.orchestra.run.smithi150.stderr: 92.9% -- replaced with /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-10T18:31:44.779 INFO:teuthology.orchestra.run.smithi115.stderr: 92.0% -- replaced with /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-10T18:31:44.795 INFO:teuthology.orchestra.run.smithi150.stderr: 93.0% -- replaced with /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-10T18:31:44.824 INFO:teuthology.orchestra.run.smithi150.stderr: 93.2% -- replaced with /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-10T18:31:44.893 INFO:teuthology.orchestra.run.smithi115.stderr: 92.8% -- replaced with /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-10T18:31:44.909 INFO:teuthology.orchestra.run.smithi150.stderr: 93.2% -- replaced with /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-10T18:31:44.911 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-09-10T18:31:44.911 INFO:teuthology.orchestra.run.smithi150.stderr:real 0m0.442s 2024-09-10T18:31:44.911 INFO:teuthology.orchestra.run.smithi150.stderr:user 0m1.600s 2024-09-10T18:31:44.911 INFO:teuthology.orchestra.run.smithi150.stderr:sys 0m0.081s 2024-09-10T18:31:44.944 INFO:teuthology.orchestra.run.smithi115.stderr: 93.0% -- replaced with /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-10T18:31:44.967 INFO:teuthology.orchestra.run.smithi115.stderr: 93.1% -- replaced with /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-10T18:31:45.039 INFO:teuthology.orchestra.run.smithi115.stderr: 93.3% -- replaced with /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-10T18:31:45.053 INFO:teuthology.orchestra.run.smithi115.stderr: 91.1% -- replaced with /var/log/ceph/a600e180-6fa0-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-10T18:31:45.055 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-09-10T18:31:45.055 INFO:teuthology.orchestra.run.smithi115.stderr:real 0m0.587s 2024-09-10T18:31:45.055 INFO:teuthology.orchestra.run.smithi115.stderr:user 0m2.894s 2024-09-10T18:31:45.055 INFO:teuthology.orchestra.run.smithi115.stderr:sys 0m0.128s 2024-09-10T18:31:45.056 INFO:tasks.cephadm:Archiving logs... 2024-09-10T18:31:45.056 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900210/remote/smithi115/log 2024-09-10T18:31:45.057 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T18:31:45.494 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900210/remote/smithi150/log 2024-09-10T18:31:45.495 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T18:31:45.760 INFO:tasks.cephadm:Removing cluster... 2024-09-10T18:31:45.761 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm rm-cluster --fsid a600e180-6fa0-11ef-bcea-c7b262605968 --force 2024-09-10T18:31:46.226 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm rm-cluster --fsid a600e180-6fa0-11ef-bcea-c7b262605968 --force 2024-09-10T18:31:46.707 INFO:tasks.cephadm:Teardown complete 2024-09-10T18:31:46.707 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T18:31:46.718 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T18:31:46.718 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T18:31:46.721 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T18:31:46.797 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T18:31:46.809 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T18:31:46.809 DEBUG:teuthology.orchestra.run.smithi115:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T18:31:46.811 DEBUG:teuthology.orchestra.run.smithi150:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T18:31:46.824 INFO:teuthology.orchestra.run.smithi115.stderr:bash: line 1: ntpq: command not found 2024-09-10T18:31:46.825 INFO:teuthology.orchestra.run.smithi150.stderr:bash: line 1: ntpq: command not found 2024-09-10T18:31:46.828 INFO:teuthology.orchestra.run.smithi115.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T18:31:46.829 INFO:teuthology.orchestra.run.smithi115.stdout:=============================================================================== 2024-09-10T18:31:46.829 INFO:teuthology.orchestra.run.smithi115.stdout:^* hv01.front.sepia.ceph.com 3 6 377 13 +80us[ +82us] +/- 40ms 2024-09-10T18:31:46.829 INFO:teuthology.orchestra.run.smithi115.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 14 +724us[ +726us] +/- 54ms 2024-09-10T18:31:46.829 INFO:teuthology.orchestra.run.smithi115.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 14 -1050us[-1048us] +/- 68ms 2024-09-10T18:31:46.829 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:31:46.830 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T18:31:46.830 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-09-10T18:31:46.830 INFO:teuthology.orchestra.run.smithi150.stdout:^* hv01.front.sepia.ceph.com 3 6 377 12 +86us[ +103us] +/- 40ms 2024-09-10T18:31:46.830 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 12 +731us[ +731us] +/- 54ms 2024-09-10T18:31:46.830 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 -1036us[-1036us] +/- 68ms 2024-09-10T18:31:46.831 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:31:46.831 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T18:31:46.840 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T18:31:46.840 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T18:31:46.861 DEBUG:teuthology.orchestra.run.smithi115:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T18:31:46.949 DEBUG:teuthology.orchestra.run.smithi150:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T18:31:47.022 DEBUG:teuthology.orchestra.run.smithi115:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T18:31:47.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:31:47.051 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T18:31:47.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:31:47.081 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T18:31:47.091 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T18:31:47.112 INFO:teuthology.task.internal:Duration was 1341.244377 seconds 2024-09-10T18:31:47.112 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T18:31:47.144 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T18:31:47.145 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T18:31:47.147 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T18:31:47.184 INFO:teuthology.orchestra.run.smithi115.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T18:31:47.186 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T18:31:47.522 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T18:31:47.522 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi115.front.sepia.ceph.com 2024-09-10T18:31:47.522 DEBUG:teuthology.orchestra.run.smithi115:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T18:31:47.582 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi150.front.sepia.ceph.com 2024-09-10T18:31:47.582 DEBUG:teuthology.orchestra.run.smithi150:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T18:31:47.609 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T18:31:47.610 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T18:31:47.625 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T18:31:47.676 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T18:31:47.676 DEBUG:teuthology.orchestra.run.smithi115:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T18:31:47.694 DEBUG:teuthology.orchestra.run.smithi150:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T18:31:47.970 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T18:31:47.980 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T18:31:47.980 DEBUG:teuthology.orchestra.run.smithi115:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T18:31:48.006 DEBUG:teuthology.orchestra.run.smithi150:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T18:31:48.031 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T18:31:48.041 DEBUG:teuthology.orchestra.run.smithi115:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T18:31:48.048 DEBUG:teuthology.orchestra.run.smithi150:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T18:31:48.069 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = core 2024-09-10T18:31:48.094 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = core 2024-09-10T18:31:48.112 DEBUG:teuthology.orchestra.run.smithi115:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T18:31:48.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:31:48.148 DEBUG:teuthology.orchestra.run.smithi150:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T18:31:48.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:31:48.167 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T18:31:48.176 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T18:31:48.177 DEBUG:teuthology.misc:Transferring archived files from smithi115:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900210/remote/smithi115 2024-09-10T18:31:48.177 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T18:31:48.228 DEBUG:teuthology.misc:Transferring archived files from smithi150:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900210/remote/smithi150 2024-09-10T18:31:48.229 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T18:31:48.265 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T18:31:48.265 DEBUG:teuthology.orchestra.run.smithi115:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T18:31:48.268 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T18:31:48.314 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T18:31:48.323 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T18:31:48.324 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T18:31:48.345 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T18:31:48.345 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T18:31:48.348 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T18:31:48.360 INFO:teuthology.orchestra.run.smithi115.stdout: 266054 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 18:31 /home/ubuntu/cephtest 2024-09-10T18:31:48.369 INFO:teuthology.orchestra.run.smithi150.stdout: 266063 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 18:31 /home/ubuntu/cephtest 2024-09-10T18:31:48.370 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T18:31:48.378 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T18:31:48.448 INFO:teuthology.run:Summary data: description: orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rotate-keys} duration: 1341.244376897812 flavor: default owner: scheduled_gabrioux@teuthology success: true 2024-09-10T18:31:48.448 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T18:31:48.530 INFO:teuthology.run:pass