2024-09-17T08:15:59.872 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-17T08:15:59.887 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T08:15:59.969 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908211 branch: reef description: orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rotate-keys} email: ceph-qa@ceph.com first_in_suite: false job_id: '7908211' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: reef ceph: conf: global: mon election default strategy: 1 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: cephadm-package install: ceph: flavor: default sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 extra_packages: - cephadm selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: reef sha1: 1cbdc9211df088cccbd94423c7cf3a2b85fe432b owner: scheduled_teuthology@teuthology priority: 930 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 5234 sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 sleep_before_teardown: 0 subset: 46/64 suite: orch suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph_1cbdc9211df088cccbd94423c7cf3a2b85fe432b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 1cbdc9211df088cccbd94423c7cf3a2b85fe432b targets: smithi118.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFvH38lqyl6hX0I8MsKcTweJrC5bMdVDKiDs8nEhxvRgwpJz8E0bhrsHnTptUa4DH/Q8RWzIha7V3QWcrgwOf4U= smithi170.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOOEglssTibasezjl2YcIaMPBpcZ3bQcW8l4vkEcN3ah4jDT7+/NtTPpzCha3nhxD3K3aNYMjnQYaBtEEIg9334= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install runc nvmetcli nvme-cli -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - cephadm.shell: mon.a: - "set -ex\nfor f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x\n\ do\n echo \"rotating key for $f\"\n K=$(ceph auth get-key $f)\n NK=\"\ $K\"\n ceph orch daemon rotate-key $f\n while [ \"$K\" == \"$NK\" ]; do\n\ \ sleep 5\n NK=$(ceph auth get-key $f)\n done\ndone\n" teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-16_22:08:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-17T08:15:59.970 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_1cbdc9211df088cccbd94423c7cf3a2b85fe432b/qa; will attempt to use it 2024-09-17T08:15:59.970 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_1cbdc9211df088cccbd94423c7cf3a2b85fe432b/qa/tasks 2024-09-17T08:15:59.970 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-17T08:15:59.972 INFO:teuthology.task.internal:Checking packages... 2024-09-17T08:15:59.992 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9' 2024-09-17T08:15:59.992 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-17T08:15:59.992 INFO:teuthology.packaging:ref: None 2024-09-17T08:15:59.993 INFO:teuthology.packaging:tag: None 2024-09-17T08:15:59.993 INFO:teuthology.packaging:branch: reef 2024-09-17T08:15:59.993 INFO:teuthology.packaging:sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-17T08:15:59.993 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-09-17T08:16:00.261 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-855.g19e2a2be 2024-09-17T08:16:00.263 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-17T08:16:00.270 INFO:teuthology.task.internal:no buildpackages task found 2024-09-17T08:16:00.270 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-17T08:16:00.316 INFO:teuthology.task.internal:Saving configuration 2024-09-17T08:16:00.327 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-17T08:16:00.332 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-17T08:16:00.357 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi118.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908211', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 08:10:25.707113', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFvH38lqyl6hX0I8MsKcTweJrC5bMdVDKiDs8nEhxvRgwpJz8E0bhrsHnTptUa4DH/Q8RWzIha7V3QWcrgwOf4U='} 2024-09-17T08:16:00.377 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi170.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908211', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 08:10:25.708335', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOOEglssTibasezjl2YcIaMPBpcZ3bQcW8l4vkEcN3ah4jDT7+/NtTPpzCha3nhxD3K3aNYMjnQYaBtEEIg9334='} 2024-09-17T08:16:00.377 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-17T08:16:00.384 INFO:teuthology.task.internal:roles: ubuntu@smithi118.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-17T08:16:00.384 INFO:teuthology.task.internal:roles: ubuntu@smithi170.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-09-17T08:16:00.384 INFO:teuthology.run_tasks:Running task console_log... 2024-09-17T08:16:00.524 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa675c9bac0>, signals=[15]) 2024-09-17T08:16:00.524 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-17T08:16:00.530 INFO:teuthology.task.internal:Opening connections... 2024-09-17T08:16:00.530 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi118.front.sepia.ceph.com 2024-09-17T08:16:00.532 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T08:16:00.608 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi170.front.sepia.ceph.com 2024-09-17T08:16:00.608 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T08:16:00.680 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-17T08:16:00.686 DEBUG:teuthology.orchestra.run.smithi118:> uname -m 2024-09-17T08:16:00.704 INFO:teuthology.orchestra.run.smithi118.stdout:x86_64 2024-09-17T08:16:00.704 DEBUG:teuthology.orchestra.run.smithi118:> cat /etc/os-release 2024-09-17T08:16:00.761 INFO:teuthology.orchestra.run.smithi118.stdout:NAME="CentOS Stream" 2024-09-17T08:16:00.761 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION="9" 2024-09-17T08:16:00.761 INFO:teuthology.orchestra.run.smithi118.stdout:ID="centos" 2024-09-17T08:16:00.761 INFO:teuthology.orchestra.run.smithi118.stdout:ID_LIKE="rhel fedora" 2024-09-17T08:16:00.761 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION_ID="9" 2024-09-17T08:16:00.761 INFO:teuthology.orchestra.run.smithi118.stdout:PLATFORM_ID="platform:el9" 2024-09-17T08:16:00.761 INFO:teuthology.orchestra.run.smithi118.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T08:16:00.761 INFO:teuthology.orchestra.run.smithi118.stdout:ANSI_COLOR="0;31" 2024-09-17T08:16:00.762 INFO:teuthology.orchestra.run.smithi118.stdout:LOGO="fedora-logo-icon" 2024-09-17T08:16:00.762 INFO:teuthology.orchestra.run.smithi118.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T08:16:00.762 INFO:teuthology.orchestra.run.smithi118.stdout:HOME_URL="https://centos.org/" 2024-09-17T08:16:00.762 INFO:teuthology.orchestra.run.smithi118.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T08:16:00.762 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T08:16:00.762 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T08:16:00.763 INFO:teuthology.lock.ops:Updating smithi118.front.sepia.ceph.com on lock server 2024-09-17T08:16:00.790 DEBUG:teuthology.orchestra.run.smithi170:> uname -m 2024-09-17T08:16:00.809 INFO:teuthology.orchestra.run.smithi170.stdout:x86_64 2024-09-17T08:16:00.809 DEBUG:teuthology.orchestra.run.smithi170:> cat /etc/os-release 2024-09-17T08:16:00.867 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="CentOS Stream" 2024-09-17T08:16:00.868 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="9" 2024-09-17T08:16:00.868 INFO:teuthology.orchestra.run.smithi170.stdout:ID="centos" 2024-09-17T08:16:00.868 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE="rhel fedora" 2024-09-17T08:16:00.868 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="9" 2024-09-17T08:16:00.868 INFO:teuthology.orchestra.run.smithi170.stdout:PLATFORM_ID="platform:el9" 2024-09-17T08:16:00.868 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T08:16:00.868 INFO:teuthology.orchestra.run.smithi170.stdout:ANSI_COLOR="0;31" 2024-09-17T08:16:00.868 INFO:teuthology.orchestra.run.smithi170.stdout:LOGO="fedora-logo-icon" 2024-09-17T08:16:00.868 INFO:teuthology.orchestra.run.smithi170.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T08:16:00.868 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://centos.org/" 2024-09-17T08:16:00.868 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T08:16:00.868 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T08:16:00.868 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T08:16:00.869 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2024-09-17T08:16:00.890 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-17T08:16:00.901 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-17T08:16:01.097 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-17T08:16:01.097 DEBUG:teuthology.orchestra.run.smithi118:> test '!' -e /home/ubuntu/cephtest 2024-09-17T08:16:01.100 DEBUG:teuthology.orchestra.run.smithi170:> test '!' -e /home/ubuntu/cephtest 2024-09-17T08:16:01.117 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-17T08:16:01.163 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-17T08:16:01.163 DEBUG:teuthology.orchestra.run.smithi118:> test -z $(ls -A /var/lib/ceph) 2024-09-17T08:16:01.165 DEBUG:teuthology.orchestra.run.smithi170:> test -z $(ls -A /var/lib/ceph) 2024-09-17T08:16:01.193 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-17T08:16:01.304 INFO:teuthology.run_tasks:Running task kernel... 2024-09-17T08:16:01.357 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-17T08:16:01.357 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-17T08:16:01.357 DEBUG:teuthology.orchestra.run.smithi118:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T08:16:01.358 DEBUG:teuthology.orchestra.run.smithi170:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T08:16:01.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:01.375 DEBUG:teuthology.orchestra.run.smithi118:> uname -r 2024-09-17T08:16:01.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:01.376 DEBUG:teuthology.orchestra.run.smithi170:> uname -r 2024-09-17T08:16:01.432 INFO:teuthology.orchestra.run.smithi118.stdout:5.14.0-503.el9.x86_64 2024-09-17T08:16:01.433 INFO:teuthology.task.kernel:Running kernel on smithi118: 5.14.0-503.el9.x86_64 2024-09-17T08:16:01.433 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum install -y kernel 2024-09-17T08:16:01.434 INFO:teuthology.orchestra.run.smithi170.stdout:5.14.0-503.el9.x86_64 2024-09-17T08:16:01.434 INFO:teuthology.task.kernel:Running kernel on smithi170: 5.14.0-503.el9.x86_64 2024-09-17T08:16:01.434 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum install -y kernel 2024-09-17T08:16:03.788 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.3 MB 00:00 2024-09-17T08:16:04.143 INFO:teuthology.orchestra.run.smithi118.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.3 MB 00:00 2024-09-17T08:16:05.894 INFO:teuthology.orchestra.run.smithi118.stdout:CentOS Stream 9 - AppStream 29 MB/s | 20 MB 00:00 2024-09-17T08:16:12.861 INFO:teuthology.orchestra.run.smithi118.stdout:CentOS Stream 9 - CRB 2.0 MB/s | 6.5 MB 00:03 2024-09-17T08:16:14.361 INFO:teuthology.orchestra.run.smithi118.stdout:CentOS Stream 9 - Extras packages 69 kB/s | 19 kB 00:00 2024-09-17T08:16:15.274 INFO:teuthology.orchestra.run.smithi118.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-17T08:16:20.671 INFO:teuthology.orchestra.run.smithi118.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2024-09-17T08:16:22.603 INFO:teuthology.orchestra.run.smithi118.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T08:16:22.603 INFO:teuthology.orchestra.run.smithi118.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T08:16:22.701 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2024-09-17T08:16:22.711 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:16:22.711 INFO:teuthology.orchestra.run.smithi118.stdout: Package Architecture Version Repository Size 2024-09-17T08:16:22.712 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:16:22.712 INFO:teuthology.orchestra.run.smithi118.stdout:Installing: 2024-09-17T08:16:22.712 INFO:teuthology.orchestra.run.smithi118.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T08:16:22.712 INFO:teuthology.orchestra.run.smithi118.stdout:Installing dependencies: 2024-09-17T08:16:22.712 INFO:teuthology.orchestra.run.smithi118.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T08:16:22.712 INFO:teuthology.orchestra.run.smithi118.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T08:16:22.712 INFO:teuthology.orchestra.run.smithi118.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T08:16:22.712 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:16:22.712 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction Summary 2024-09-17T08:16:22.712 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:16:22.712 INFO:teuthology.orchestra.run.smithi118.stdout:Install 4 Packages 2024-09-17T08:16:22.712 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:16:22.713 INFO:teuthology.orchestra.run.smithi118.stdout:Total download size: 79 M 2024-09-17T08:16:22.713 INFO:teuthology.orchestra.run.smithi118.stdout:Installed size: 126 M 2024-09-17T08:16:22.713 INFO:teuthology.orchestra.run.smithi118.stdout:Downloading Packages: 2024-09-17T08:16:22.814 INFO:teuthology.orchestra.run.smithi118.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 443 kB/s | 36 kB 00:00 2024-09-17T08:16:22.961 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 20 MB 00:18 2024-09-17T08:16:26.089 INFO:teuthology.orchestra.run.smithi118.stdout:(2/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 10 MB/s | 35 MB 00:03 2024-09-17T08:16:26.697 INFO:teuthology.orchestra.run.smithi118.stdout:(3/4): kernel-core-5.14.0-508.el9.x86_64.rpm 3.9 MB/s | 16 MB 00:03 2024-09-17T08:16:28.187 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - CRB 4.5 MB/s | 6.5 MB 00:01 2024-09-17T08:16:29.646 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - Extras packages 79 kB/s | 19 kB 00:00 2024-09-17T08:16:30.356 INFO:teuthology.orchestra.run.smithi118.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 3.8 MB/s | 29 MB 00:07 2024-09-17T08:16:30.356 INFO:teuthology.orchestra.run.smithi118.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:16:30.356 INFO:teuthology.orchestra.run.smithi118.stdout:Total 10 MB/s | 79 MB 00:07 2024-09-17T08:16:30.567 INFO:teuthology.orchestra.run.smithi170.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-17T08:16:31.040 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction check 2024-09-17T08:16:31.377 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction check succeeded. 2024-09-17T08:16:31.377 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction test 2024-09-17T08:16:32.340 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction test succeeded. 2024-09-17T08:16:32.340 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction 2024-09-17T08:16:34.789 INFO:teuthology.orchestra.run.smithi118.stdout: Preparing : 1/1 2024-09-17T08:16:35.452 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T08:16:35.540 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T08:16:36.022 INFO:teuthology.orchestra.run.smithi170.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-17T08:16:37.946 INFO:teuthology.orchestra.run.smithi170.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T08:16:37.946 INFO:teuthology.orchestra.run.smithi170.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T08:16:38.034 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-09-17T08:16:38.046 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:16:38.046 INFO:teuthology.orchestra.run.smithi170.stdout: Package Architecture Version Repository Size 2024-09-17T08:16:38.046 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:16:38.046 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2024-09-17T08:16:38.046 INFO:teuthology.orchestra.run.smithi170.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T08:16:38.046 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2024-09-17T08:16:38.046 INFO:teuthology.orchestra.run.smithi170.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T08:16:38.046 INFO:teuthology.orchestra.run.smithi170.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T08:16:38.046 INFO:teuthology.orchestra.run.smithi170.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T08:16:38.046 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:16:38.047 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2024-09-17T08:16:38.047 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:16:38.047 INFO:teuthology.orchestra.run.smithi170.stdout:Install 4 Packages 2024-09-17T08:16:38.047 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:16:38.047 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 79 M 2024-09-17T08:16:38.047 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 126 M 2024-09-17T08:16:38.047 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2024-09-17T08:16:38.283 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T08:16:38.399 INFO:teuthology.orchestra.run.smithi170.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 337 kB/s | 36 kB 00:00 2024-09-17T08:16:38.419 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T08:16:38.708 INFO:teuthology.orchestra.run.smithi170.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-09-17T08:16:39.858 INFO:teuthology.orchestra.run.smithi170.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 20 MB/s | 29 MB 00:01 2024-09-17T08:16:41.516 INFO:teuthology.orchestra.run.smithi170.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 11 MB/s | 35 MB 00:03 2024-09-17T08:16:41.517 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:16:41.517 INFO:teuthology.orchestra.run.smithi170.stdout:Total 23 MB/s | 79 MB 00:03 2024-09-17T08:16:42.200 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2024-09-17T08:16:42.544 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2024-09-17T08:16:42.544 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2024-09-17T08:16:43.254 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T08:16:43.303 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T08:16:43.494 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2024-09-17T08:16:43.495 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2024-09-17T08:16:45.923 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2024-09-17T08:16:46.642 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T08:16:46.725 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T08:16:48.163 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T08:16:49.481 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T08:16:49.613 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T08:16:54.426 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T08:16:54.547 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T08:16:59.387 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T08:17:26.994 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T08:17:27.033 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T08:17:27.429 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T08:17:27.429 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T08:17:27.429 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T08:17:27.429 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T08:17:28.292 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T08:17:28.292 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:17:28.292 INFO:teuthology.orchestra.run.smithi118.stdout:Installed: 2024-09-17T08:17:28.292 INFO:teuthology.orchestra.run.smithi118.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T08:17:28.292 INFO:teuthology.orchestra.run.smithi118.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T08:17:28.292 INFO:teuthology.orchestra.run.smithi118.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T08:17:28.292 INFO:teuthology.orchestra.run.smithi118.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T08:17:28.292 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:17:28.293 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2024-09-17T08:17:28.697 DEBUG:teuthology.orchestra.run.smithi118:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T08:17:28.780 INFO:teuthology.orchestra.run.smithi118.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T08:17:28.780 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T08:17:28.781 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T08:17:28.781 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T08:17:28.781 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T08:17:28.781 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-17T08:17:28.781 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi118.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T08:17:28.781 DEBUG:teuthology.orchestra.run.smithi118:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T08:17:28.840 INFO:teuthology.orchestra.run.smithi118.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T08:17:28.840 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T08:17:28.840 DEBUG:teuthology.orchestra.run.smithi118:> sudo rpm -qi grub2-tools 2024-09-17T08:17:28.877 INFO:teuthology.orchestra.run.smithi118.stdout:Name : grub2-tools 2024-09-17T08:17:28.877 INFO:teuthology.orchestra.run.smithi118.stdout:Epoch : 1 2024-09-17T08:17:28.877 INFO:teuthology.orchestra.run.smithi118.stdout:Version : 2.06 2024-09-17T08:17:28.877 INFO:teuthology.orchestra.run.smithi118.stdout:Release : 82.el9 2024-09-17T08:17:28.877 INFO:teuthology.orchestra.run.smithi118.stdout:Architecture: x86_64 2024-09-17T08:17:28.877 INFO:teuthology.orchestra.run.smithi118.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T08:17:28.877 INFO:teuthology.orchestra.run.smithi118.stdout:Group : Unspecified 2024-09-17T08:17:28.878 INFO:teuthology.orchestra.run.smithi118.stdout:Size : 8274670 2024-09-17T08:17:28.878 INFO:teuthology.orchestra.run.smithi118.stdout:License : GPLv3+ 2024-09-17T08:17:28.878 INFO:teuthology.orchestra.run.smithi118.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T08:17:28.878 INFO:teuthology.orchestra.run.smithi118.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T08:17:28.878 INFO:teuthology.orchestra.run.smithi118.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T08:17:28.878 INFO:teuthology.orchestra.run.smithi118.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T08:17:28.878 INFO:teuthology.orchestra.run.smithi118.stdout:Packager : builder@centos.org 2024-09-17T08:17:28.878 INFO:teuthology.orchestra.run.smithi118.stdout:Vendor : CentOS 2024-09-17T08:17:28.878 INFO:teuthology.orchestra.run.smithi118.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T08:17:28.878 INFO:teuthology.orchestra.run.smithi118.stdout:Summary : Support tools for GRUB. 2024-09-17T08:17:28.878 INFO:teuthology.orchestra.run.smithi118.stdout:Description : 2024-09-17T08:17:28.878 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:17:28.878 INFO:teuthology.orchestra.run.smithi118.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T08:17:28.878 INFO:teuthology.orchestra.run.smithi118.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T08:17:28.879 INFO:teuthology.orchestra.run.smithi118.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T08:17:28.879 INFO:teuthology.orchestra.run.smithi118.stdout:hardware devices. 2024-09-17T08:17:28.879 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:17:28.879 INFO:teuthology.orchestra.run.smithi118.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T08:17:28.880 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T08:17:28.880 INFO:teuthology.task.kernel:Updating grub on smithi118 to boot 5.14.0-508.el9.x86_64 2024-09-17T08:17:28.880 DEBUG:teuthology.orchestra.run.smithi118:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T08:17:29.563 INFO:teuthology.orchestra.run.smithi118.stderr:Generating grub configuration file ... 2024-09-17T08:17:30.878 INFO:teuthology.orchestra.run.smithi118.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T08:17:30.910 INFO:teuthology.orchestra.run.smithi118.stderr:done 2024-09-17T08:17:30.913 DEBUG:teuthology.orchestra.run.smithi118:> mktemp 2024-09-17T08:17:30.929 INFO:teuthology.orchestra.run.smithi118.stdout:/tmp/tmp.0rkYMPikKK 2024-09-17T08:17:30.929 DEBUG:teuthology.orchestra.run.smithi118:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.0rkYMPikKK 2024-09-17T08:17:30.995 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 0666 /tmp/tmp.0rkYMPikKK 2024-09-17T08:17:31.138 DEBUG:teuthology.orchestra.remote:smithi118:/tmp/tmp.0rkYMPikKK is 6KB 2024-09-17T08:17:31.187 DEBUG:teuthology.orchestra.run.smithi118:> rm -fr /tmp/tmp.0rkYMPikKK 2024-09-17T08:17:31.201 DEBUG:teuthology.orchestra.run.smithi118:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T08:17:31.264 INFO:teuthology.orchestra.run.smithi118.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T08:17:31.264 INFO:teuthology.orchestra.run.smithi118.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T08:17:31.264 INFO:teuthology.orchestra.run.smithi118.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T08:17:31.264 INFO:teuthology.orchestra.run.smithi118.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T08:17:31.265 DEBUG:teuthology.orchestra.run.smithi118:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T08:17:31.440 DEBUG:teuthology.orchestra.run.smithi118:> sudo shutdown -r now 2024-09-17T08:17:37.905 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T08:17:37.953 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T08:17:38.329 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T08:17:38.329 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T08:17:38.329 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T08:17:38.329 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T08:17:39.186 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T08:17:39.186 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:17:39.187 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2024-09-17T08:17:39.187 INFO:teuthology.orchestra.run.smithi170.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T08:17:39.187 INFO:teuthology.orchestra.run.smithi170.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T08:17:39.187 INFO:teuthology.orchestra.run.smithi170.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T08:17:39.187 INFO:teuthology.orchestra.run.smithi170.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T08:17:39.187 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:17:39.187 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-09-17T08:17:39.541 DEBUG:teuthology.orchestra.run.smithi170:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T08:17:39.629 INFO:teuthology.orchestra.run.smithi170.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T08:17:39.629 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T08:17:39.629 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T08:17:39.629 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T08:17:39.630 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T08:17:39.630 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-17T08:17:39.630 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi170.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T08:17:39.630 DEBUG:teuthology.orchestra.run.smithi170:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T08:17:39.688 INFO:teuthology.orchestra.run.smithi170.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T08:17:39.688 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T08:17:39.689 DEBUG:teuthology.orchestra.run.smithi170:> sudo rpm -qi grub2-tools 2024-09-17T08:17:39.728 INFO:teuthology.orchestra.run.smithi170.stdout:Name : grub2-tools 2024-09-17T08:17:39.728 INFO:teuthology.orchestra.run.smithi170.stdout:Epoch : 1 2024-09-17T08:17:39.728 INFO:teuthology.orchestra.run.smithi170.stdout:Version : 2.06 2024-09-17T08:17:39.729 INFO:teuthology.orchestra.run.smithi170.stdout:Release : 82.el9 2024-09-17T08:17:39.729 INFO:teuthology.orchestra.run.smithi170.stdout:Architecture: x86_64 2024-09-17T08:17:39.729 INFO:teuthology.orchestra.run.smithi170.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T08:17:39.729 INFO:teuthology.orchestra.run.smithi170.stdout:Group : Unspecified 2024-09-17T08:17:39.729 INFO:teuthology.orchestra.run.smithi170.stdout:Size : 8274670 2024-09-17T08:17:39.730 INFO:teuthology.orchestra.run.smithi170.stdout:License : GPLv3+ 2024-09-17T08:17:39.730 INFO:teuthology.orchestra.run.smithi170.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T08:17:39.730 INFO:teuthology.orchestra.run.smithi170.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T08:17:39.730 INFO:teuthology.orchestra.run.smithi170.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T08:17:39.730 INFO:teuthology.orchestra.run.smithi170.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T08:17:39.731 INFO:teuthology.orchestra.run.smithi170.stdout:Packager : builder@centos.org 2024-09-17T08:17:39.731 INFO:teuthology.orchestra.run.smithi170.stdout:Vendor : CentOS 2024-09-17T08:17:39.731 INFO:teuthology.orchestra.run.smithi170.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T08:17:39.731 INFO:teuthology.orchestra.run.smithi170.stdout:Summary : Support tools for GRUB. 2024-09-17T08:17:39.731 INFO:teuthology.orchestra.run.smithi170.stdout:Description : 2024-09-17T08:17:39.731 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:17:39.731 INFO:teuthology.orchestra.run.smithi170.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T08:17:39.731 INFO:teuthology.orchestra.run.smithi170.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T08:17:39.732 INFO:teuthology.orchestra.run.smithi170.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T08:17:39.732 INFO:teuthology.orchestra.run.smithi170.stdout:hardware devices. 2024-09-17T08:17:39.732 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:17:39.732 INFO:teuthology.orchestra.run.smithi170.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T08:17:39.733 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T08:17:39.733 INFO:teuthology.task.kernel:Updating grub on smithi170 to boot 5.14.0-508.el9.x86_64 2024-09-17T08:17:39.733 DEBUG:teuthology.orchestra.run.smithi170:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T08:17:40.377 INFO:teuthology.orchestra.run.smithi170.stderr:Generating grub configuration file ... 2024-09-17T08:17:41.671 INFO:teuthology.orchestra.run.smithi170.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T08:17:41.696 INFO:teuthology.orchestra.run.smithi170.stderr:done 2024-09-17T08:17:41.699 DEBUG:teuthology.orchestra.run.smithi170:> mktemp 2024-09-17T08:17:41.714 INFO:teuthology.orchestra.run.smithi170.stdout:/tmp/tmp.eqhPvy83fB 2024-09-17T08:17:41.714 DEBUG:teuthology.orchestra.run.smithi170:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.eqhPvy83fB 2024-09-17T08:17:41.807 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 0666 /tmp/tmp.eqhPvy83fB 2024-09-17T08:17:41.904 DEBUG:teuthology.orchestra.remote:smithi170:/tmp/tmp.eqhPvy83fB is 6KB 2024-09-17T08:17:41.952 DEBUG:teuthology.orchestra.run.smithi170:> rm -fr /tmp/tmp.eqhPvy83fB 2024-09-17T08:17:41.968 DEBUG:teuthology.orchestra.run.smithi170:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T08:17:42.033 INFO:teuthology.orchestra.run.smithi170.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T08:17:42.033 INFO:teuthology.orchestra.run.smithi170.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T08:17:42.033 INFO:teuthology.orchestra.run.smithi170.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T08:17:42.033 INFO:teuthology.orchestra.run.smithi170.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T08:17:42.035 DEBUG:teuthology.orchestra.run.smithi170:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T08:17:42.188 DEBUG:teuthology.orchestra.run.smithi170:> sudo shutdown -r now 2024-09-17T08:18:01.455 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-17T08:18:01.456 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-09-17T08:18:01.457 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T08:18:12.198 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-17T08:18:12.198 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-09-17T08:18:12.199 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T08:18:35.965 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-09-17T08:18:44.973 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-09-17T08:18:44.974 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T08:19:12.226 DEBUG:teuthology.orchestra.remote:timed out 2024-09-17T08:19:21.232 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-09-17T08:19:21.233 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T08:19:21.703 DEBUG:teuthology.orchestra.run.smithi170:> true 2024-09-17T08:19:22.179 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-09-17T08:19:22.179 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-508.el9.x86_64"... 2024-09-17T08:19:22.179 DEBUG:teuthology.orchestra.run.smithi170:> uname -r 2024-09-17T08:19:22.194 INFO:teuthology.orchestra.run.smithi170.stdout:5.14.0-508.el9.x86_64 2024-09-17T08:19:22.194 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T08:19:22.194 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T08:19:22.194 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T08:19:23.195 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-17T08:19:23.195 DEBUG:teuthology.orchestra.run.smithi170:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T08:19:23.277 INFO:teuthology.orchestra.run.smithi170.stdout:ttyS1 2024-09-17T08:19:23.311 DEBUG:teuthology.parallel:result is None 2024-09-17T08:19:44.976 DEBUG:teuthology.orchestra.remote:timed out 2024-09-17T08:19:56.978 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-09-17T08:19:56.978 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T08:19:57.474 DEBUG:teuthology.orchestra.run.smithi118:> true 2024-09-17T08:19:57.841 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-09-17T08:19:57.841 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-17T08:19:57.841 DEBUG:teuthology.orchestra.run.smithi118:> uname -r 2024-09-17T08:19:57.857 INFO:teuthology.orchestra.run.smithi118.stdout:5.14.0-508.el9.x86_64 2024-09-17T08:19:57.858 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T08:19:57.858 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T08:19:57.858 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T08:19:58.858 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-17T08:19:58.859 DEBUG:teuthology.orchestra.run.smithi118:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T08:19:58.940 INFO:teuthology.orchestra.run.smithi118.stdout:ttyS1 2024-09-17T08:19:58.959 DEBUG:teuthology.parallel:result is None 2024-09-17T08:19:58.959 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-17T08:19:58.969 INFO:teuthology.task.internal:Creating test directory... 2024-09-17T08:19:58.970 DEBUG:teuthology.orchestra.run.smithi118:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T08:19:58.972 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T08:19:58.993 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-17T08:19:59.040 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-17T08:19:59.071 INFO:teuthology.task.internal:Creating archive directory... 2024-09-17T08:19:59.071 DEBUG:teuthology.orchestra.run.smithi118:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T08:19:59.073 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T08:19:59.111 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-17T08:19:59.166 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-17T08:19:59.167 DEBUG:teuthology.orchestra.run.smithi118:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T08:19:59.169 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T08:19:59.212 INFO:teuthology.orchestra.run.smithi118.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T08:19:59.212 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T08:19:59.223 INFO:teuthology.orchestra.run.smithi118.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T08:19:59.224 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T08:19:59.226 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-17T08:19:59.322 INFO:teuthology.task.internal:Configuring sudo... 2024-09-17T08:19:59.322 DEBUG:teuthology.orchestra.run.smithi118:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T08:19:59.325 DEBUG:teuthology.orchestra.run.smithi170:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T08:19:59.353 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-17T08:19:59.432 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-17T08:19:59.432 DEBUG:teuthology.orchestra.run.smithi118:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T08:19:59.435 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T08:19:59.453 DEBUG:teuthology.orchestra.run.smithi118:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T08:19:59.535 DEBUG:teuthology.orchestra.run.smithi118:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T08:19:59.580 DEBUG:teuthology.orchestra.run.smithi118:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T08:19:59.664 DEBUG:teuthology.orchestra.run.smithi118:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T08:19:59.705 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-17T08:19:59.705 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T08:19:59.770 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T08:19:59.815 DEBUG:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T08:19:59.881 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T08:19:59.924 DEBUG:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T08:19:59.967 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-17T08:19:59.967 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T08:20:00.034 DEBUG:teuthology.orchestra.run.smithi118:> sudo service rsyslog restart 2024-09-17T08:20:00.036 DEBUG:teuthology.orchestra.run.smithi170:> sudo service rsyslog restart 2024-09-17T08:20:00.082 INFO:teuthology.orchestra.run.smithi118.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T08:20:00.121 INFO:teuthology.orchestra.run.smithi170.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T08:20:00.574 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-17T08:20:00.582 INFO:teuthology.task.internal:Starting timer... 2024-09-17T08:20:00.582 INFO:teuthology.run_tasks:Running task pcp... 2024-09-17T08:20:00.654 INFO:teuthology.run_tasks:Running task selinux... 2024-09-17T08:20:00.718 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-17T08:20:00.718 DEBUG:teuthology.orchestra.run.smithi118:> sudo service auditd rotate 2024-09-17T08:20:00.806 INFO:teuthology.orchestra.run.smithi118.stdout:Rotating logs: 2024-09-17T08:20:00.808 DEBUG:teuthology.orchestra.run.smithi170:> sudo service auditd rotate 2024-09-17T08:20:00.866 INFO:teuthology.orchestra.run.smithi170.stdout:Rotating logs: 2024-09-17T08:20:00.868 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-17T08:20:00.868 DEBUG:teuthology.orchestra.run.smithi118:> /usr/sbin/getenforce 2024-09-17T08:20:00.894 INFO:teuthology.orchestra.run.smithi118.stdout:Permissive 2024-09-17T08:20:00.894 DEBUG:teuthology.orchestra.run.smithi170:> /usr/sbin/getenforce 2024-09-17T08:20:00.925 INFO:teuthology.orchestra.run.smithi170.stdout:Permissive 2024-09-17T08:20:00.925 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi118.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi170.front.sepia.ceph.com': 'permissive'} 2024-09-17T08:20:00.925 DEBUG:teuthology.orchestra.run.smithi118:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T08:20:00.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:00.961 DEBUG:teuthology.orchestra.run.smithi170:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T08:20:00.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:00.992 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-17T08:20:00.992 DEBUG:teuthology.orchestra.run.smithi118:> sudo /usr/sbin/setenforce permissive 2024-09-17T08:20:01.030 DEBUG:teuthology.orchestra.run.smithi170:> sudo /usr/sbin/setenforce permissive 2024-09-17T08:20:01.066 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-17T08:20:01.075 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-17T08:20:01.086 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-17T08:20:01.087 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi118.front.sepia.ceph.com,smithi170.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-17T08:23:48.525 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi118.front.sepia.ceph.com'), Remote(name='ubuntu@smithi170.front.sepia.ceph.com')] 2024-09-17T08:23:48.526 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-09-17T08:23:48.528 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T08:23:48.607 DEBUG:teuthology.orchestra.run.smithi118:> true 2024-09-17T08:23:48.640 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-09-17T08:23:48.641 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-09-17T08:23:48.641 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T08:23:48.715 DEBUG:teuthology.orchestra.run.smithi170:> true 2024-09-17T08:23:48.791 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-09-17T08:23:48.791 INFO:teuthology.run_tasks:Running task clock... 2024-09-17T08:23:48.802 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-17T08:23:48.803 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T08:23:48.803 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T08:23:48.805 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T08:23:48.806 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T08:23:48.832 INFO:teuthology.orchestra.run.smithi118.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T08:23:48.846 INFO:teuthology.orchestra.run.smithi118.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T08:23:48.863 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T08:23:48.871 INFO:teuthology.orchestra.run.smithi118.stderr:sudo: ntpd: command not found 2024-09-17T08:23:48.874 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T08:23:48.882 INFO:teuthology.orchestra.run.smithi118.stdout:506 Cannot talk to daemon 2024-09-17T08:23:48.896 INFO:teuthology.orchestra.run.smithi118.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T08:23:48.899 INFO:teuthology.orchestra.run.smithi170.stderr:sudo: ntpd: command not found 2024-09-17T08:23:48.910 INFO:teuthology.orchestra.run.smithi118.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T08:23:48.910 INFO:teuthology.orchestra.run.smithi170.stdout:506 Cannot talk to daemon 2024-09-17T08:23:48.925 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T08:23:48.939 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T08:23:48.967 INFO:teuthology.orchestra.run.smithi118.stderr:bash: line 1: ntpq: command not found 2024-09-17T08:23:48.971 INFO:teuthology.orchestra.run.smithi118.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T08:23:48.971 INFO:teuthology.orchestra.run.smithi118.stdout:=============================================================================== 2024-09-17T08:23:48.971 INFO:teuthology.orchestra.run.smithi118.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T08:23:48.971 INFO:teuthology.orchestra.run.smithi118.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T08:23:48.971 INFO:teuthology.orchestra.run.smithi118.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T08:23:48.971 INFO:teuthology.orchestra.run.smithi118.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T08:23:48.991 INFO:teuthology.orchestra.run.smithi170.stderr:bash: line 1: ntpq: command not found 2024-09-17T08:23:48.995 INFO:teuthology.orchestra.run.smithi170.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T08:23:48.996 INFO:teuthology.orchestra.run.smithi170.stdout:=============================================================================== 2024-09-17T08:23:48.996 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T08:23:48.996 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T08:23:48.996 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T08:23:48.996 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T08:23:48.997 INFO:teuthology.run_tasks:Running task pexec... 2024-09-17T08:23:49.007 INFO:teuthology.task.pexec:Executing custom commands... 2024-09-17T08:23:49.007 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi118.front.sepia.ceph.com 2024-09-17T08:23:49.007 DEBUG:teuthology.orchestra.run.smithi118:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-17T08:23:49.008 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi170.front.sepia.ceph.com 2024-09-17T08:23:49.008 DEBUG:teuthology.orchestra.run.smithi170:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-17T08:23:49.362 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2024-09-17T08:23:49.373 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:23:49.373 INFO:teuthology.orchestra.run.smithi118.stdout: Package Architecture Version Repository Size 2024-09-17T08:23:49.373 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:23:49.373 INFO:teuthology.orchestra.run.smithi118.stdout:Removing: 2024-09-17T08:23:49.374 INFO:teuthology.orchestra.run.smithi118.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-17T08:23:49.374 INFO:teuthology.orchestra.run.smithi118.stdout:Removing unused dependencies: 2024-09-17T08:23:49.374 INFO:teuthology.orchestra.run.smithi118.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-17T08:23:49.374 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:23:49.374 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction Summary 2024-09-17T08:23:49.374 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:23:49.374 INFO:teuthology.orchestra.run.smithi118.stdout:Remove 2 Packages 2024-09-17T08:23:49.374 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:23:49.374 INFO:teuthology.orchestra.run.smithi118.stdout:Freed space: 5.5 M 2024-09-17T08:23:49.374 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction check 2024-09-17T08:23:49.378 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction check succeeded. 2024-09-17T08:23:49.378 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction test 2024-09-17T08:23:49.382 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-09-17T08:23:49.400 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:23:49.401 INFO:teuthology.orchestra.run.smithi170.stdout: Package Architecture Version Repository Size 2024-09-17T08:23:49.401 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:23:49.401 INFO:teuthology.orchestra.run.smithi170.stdout:Removing: 2024-09-17T08:23:49.401 INFO:teuthology.orchestra.run.smithi170.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-17T08:23:49.401 INFO:teuthology.orchestra.run.smithi170.stdout:Removing unused dependencies: 2024-09-17T08:23:49.401 INFO:teuthology.orchestra.run.smithi170.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-17T08:23:49.401 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:23:49.401 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2024-09-17T08:23:49.401 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:23:49.401 INFO:teuthology.orchestra.run.smithi170.stdout:Remove 2 Packages 2024-09-17T08:23:49.401 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:23:49.401 INFO:teuthology.orchestra.run.smithi170.stdout:Freed space: 5.5 M 2024-09-17T08:23:49.401 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2024-09-17T08:23:49.405 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2024-09-17T08:23:49.405 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2024-09-17T08:23:49.441 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction test succeeded. 2024-09-17T08:23:49.441 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction 2024-09-17T08:23:49.472 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2024-09-17T08:23:49.472 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2024-09-17T08:23:49.579 INFO:teuthology.orchestra.run.smithi118.stdout: Preparing : 1/1 2024-09-17T08:23:49.608 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2024-09-17T08:23:49.648 INFO:teuthology.orchestra.run.smithi118.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-17T08:23:49.695 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-17T08:23:49.704 INFO:teuthology.orchestra.run.smithi118.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-17T08:23:49.754 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-17T08:23:50.114 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-17T08:23:50.114 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-17T08:23:50.266 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-17T08:23:50.266 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-17T08:23:50.380 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-17T08:23:50.380 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:23:50.380 INFO:teuthology.orchestra.run.smithi118.stdout:Removed: 2024-09-17T08:23:50.380 INFO:teuthology.orchestra.run.smithi118.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-17T08:23:50.380 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:23:50.380 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2024-09-17T08:23:50.567 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-17T08:23:50.567 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:23:50.567 INFO:teuthology.orchestra.run.smithi170.stdout:Removed: 2024-09-17T08:23:50.567 INFO:teuthology.orchestra.run.smithi170.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-17T08:23:50.567 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:23:50.567 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-09-17T08:23:51.260 INFO:teuthology.orchestra.run.smithi118.stdout:Last metadata expiration check: 0:00:50 ago on Tue 17 Sep 2024 08:23:00 AM UTC. 2024-09-17T08:23:51.398 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2024-09-17T08:23:51.399 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:23:51.399 INFO:teuthology.orchestra.run.smithi118.stdout: Package Arch Version Repository Size 2024-09-17T08:23:51.399 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:23:51.399 INFO:teuthology.orchestra.run.smithi118.stdout:Installing: 2024-09-17T08:23:51.399 INFO:teuthology.orchestra.run.smithi118.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-17T08:23:51.399 INFO:teuthology.orchestra.run.smithi118.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-17T08:23:51.399 INFO:teuthology.orchestra.run.smithi118.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-17T08:23:51.399 INFO:teuthology.orchestra.run.smithi118.stdout:Installing dependencies: 2024-09-17T08:23:51.399 INFO:teuthology.orchestra.run.smithi118.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-17T08:23:51.399 INFO:teuthology.orchestra.run.smithi118.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-17T08:23:51.399 INFO:teuthology.orchestra.run.smithi118.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-17T08:23:51.399 INFO:teuthology.orchestra.run.smithi118.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-17T08:23:51.400 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:23:51.400 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction Summary 2024-09-17T08:23:51.400 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:23:51.400 INFO:teuthology.orchestra.run.smithi118.stdout:Install 7 Packages 2024-09-17T08:23:51.400 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:23:51.419 INFO:teuthology.orchestra.run.smithi118.stdout:Total download size: 5.2 M 2024-09-17T08:23:51.420 INFO:teuthology.orchestra.run.smithi118.stdout:Installed size: 19 M 2024-09-17T08:23:51.420 INFO:teuthology.orchestra.run.smithi118.stdout:Downloading Packages: 2024-09-17T08:23:51.492 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:00:51 ago on Tue 17 Sep 2024 08:23:00 AM UTC. 2024-09-17T08:23:51.631 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-09-17T08:23:51.632 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:23:51.632 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2024-09-17T08:23:51.632 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:23:51.632 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2024-09-17T08:23:51.632 INFO:teuthology.orchestra.run.smithi170.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-17T08:23:51.632 INFO:teuthology.orchestra.run.smithi170.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-17T08:23:51.632 INFO:teuthology.orchestra.run.smithi170.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-17T08:23:51.632 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2024-09-17T08:23:51.632 INFO:teuthology.orchestra.run.smithi170.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-17T08:23:51.633 INFO:teuthology.orchestra.run.smithi170.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-17T08:23:51.633 INFO:teuthology.orchestra.run.smithi170.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-17T08:23:51.633 INFO:teuthology.orchestra.run.smithi170.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-17T08:23:51.633 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:23:51.633 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2024-09-17T08:23:51.633 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:23:51.633 INFO:teuthology.orchestra.run.smithi170.stdout:Install 7 Packages 2024-09-17T08:23:51.633 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:23:51.655 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 5.2 M 2024-09-17T08:23:51.655 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 19 M 2024-09-17T08:23:51.655 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2024-09-17T08:23:51.877 INFO:teuthology.orchestra.run.smithi118.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 286 kB/s | 44 kB 00:00 2024-09-17T08:23:51.904 INFO:teuthology.orchestra.run.smithi118.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 587 kB/s | 106 kB 00:00 2024-09-17T08:23:51.930 INFO:teuthology.orchestra.run.smithi170.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 213 kB/s | 44 kB 00:00 2024-09-17T08:23:51.970 INFO:teuthology.orchestra.run.smithi118.stdout:(3/7): python3-kmod-0.9-32.el9.x86_64.rpm 1.2 MB/s | 84 kB 00:00 2024-09-17T08:23:52.004 INFO:teuthology.orchestra.run.smithi118.stdout:(4/7): python3-configshell-1.1.30-1.el9.noarch. 569 kB/s | 72 kB 00:00 2024-09-17T08:23:52.039 INFO:teuthology.orchestra.run.smithi170.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 334 kB/s | 106 kB 00:00 2024-09-17T08:23:52.054 INFO:teuthology.orchestra.run.smithi118.stdout:(5/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 2.7 MB/s | 906 kB 00:00 2024-09-17T08:23:52.064 INFO:teuthology.orchestra.run.smithi170.stdout:(3/7): python3-configshell-1.1.30-1.el9.noarch. 535 kB/s | 72 kB 00:00 2024-09-17T08:23:52.154 INFO:teuthology.orchestra.run.smithi118.stdout:(6/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 4.4 MB/s | 837 kB 00:00 2024-09-17T08:23:52.164 INFO:teuthology.orchestra.run.smithi170.stdout:(4/7): python3-kmod-0.9-32.el9.x86_64.rpm 671 kB/s | 84 kB 00:00 2024-09-17T08:23:52.365 INFO:teuthology.orchestra.run.smithi170.stdout:(5/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 1.4 MB/s | 906 kB 00:00 2024-09-17T08:23:52.507 INFO:teuthology.orchestra.run.smithi170.stdout:(6/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 1.8 MB/s | 837 kB 00:00 2024-09-17T08:23:52.630 INFO:teuthology.orchestra.run.smithi118.stdout:(7/7): runc-1.1.13-4.el9.x86_64.rpm 5.1 MB/s | 3.2 MB 00:00 2024-09-17T08:23:52.631 INFO:teuthology.orchestra.run.smithi118.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:23:52.631 INFO:teuthology.orchestra.run.smithi118.stdout:Total 4.3 MB/s | 5.2 MB 00:01 2024-09-17T08:23:52.747 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction check 2024-09-17T08:23:52.762 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction check succeeded. 2024-09-17T08:23:52.762 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction test 2024-09-17T08:23:53.086 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction test succeeded. 2024-09-17T08:23:53.087 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction 2024-09-17T08:23:53.399 INFO:teuthology.orchestra.run.smithi170.stdout:(7/7): runc-1.1.13-4.el9.x86_64.rpm 2.6 MB/s | 3.2 MB 00:01 2024-09-17T08:23:53.399 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:23:53.399 INFO:teuthology.orchestra.run.smithi170.stdout:Total 3.0 MB/s | 5.2 MB 00:01 2024-09-17T08:23:53.508 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2024-09-17T08:23:53.523 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2024-09-17T08:23:53.523 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2024-09-17T08:23:53.558 INFO:teuthology.orchestra.run.smithi118.stdout: Preparing : 1/1 2024-09-17T08:23:53.640 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-17T08:23:53.675 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-17T08:23:53.704 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-17T08:23:53.869 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2024-09-17T08:23:53.869 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2024-09-17T08:23:53.921 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-17T08:23:53.960 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-17T08:23:54.336 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2024-09-17T08:23:54.391 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-17T08:23:54.413 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-17T08:23:54.425 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-17T08:23:54.466 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-17T08:23:54.498 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-17T08:23:54.526 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-17T08:23:54.601 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-17T08:23:54.700 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-17T08:23:54.737 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-17T08:23:55.169 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-17T08:23:55.198 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-17T08:23:55.319 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-17T08:23:55.385 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-17T08:23:55.643 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-17T08:23:55.643 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-17T08:23:55.643 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-17T08:23:55.643 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-17T08:23:55.643 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-17T08:23:55.643 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-17T08:23:55.643 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-17T08:23:55.988 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-17T08:23:55.988 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:23:55.988 INFO:teuthology.orchestra.run.smithi118.stdout:Installed: 2024-09-17T08:23:55.988 INFO:teuthology.orchestra.run.smithi118.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-17T08:23:55.988 INFO:teuthology.orchestra.run.smithi118.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-17T08:23:55.988 INFO:teuthology.orchestra.run.smithi118.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-17T08:23:55.989 INFO:teuthology.orchestra.run.smithi118.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-17T08:23:55.989 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:23:55.989 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2024-09-17T08:23:56.401 DEBUG:teuthology.parallel:result is None 2024-09-17T08:23:56.440 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-17T08:23:56.440 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-17T08:23:56.440 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-17T08:23:56.440 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-17T08:23:56.440 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-17T08:23:56.440 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-17T08:23:56.440 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-17T08:23:56.773 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-17T08:23:56.774 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:23:56.774 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2024-09-17T08:23:56.774 INFO:teuthology.orchestra.run.smithi170.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-17T08:23:56.774 INFO:teuthology.orchestra.run.smithi170.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-17T08:23:56.774 INFO:teuthology.orchestra.run.smithi170.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-17T08:23:56.774 INFO:teuthology.orchestra.run.smithi170.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-17T08:23:56.774 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:23:56.774 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-09-17T08:23:57.181 DEBUG:teuthology.parallel:result is None 2024-09-17T08:23:57.182 INFO:teuthology.run_tasks:Running task install... 2024-09-17T08:23:57.190 DEBUG:teuthology.task.install:project ceph 2024-09-17T08:23:57.190 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'}, 'extra_packages': ['cephadm']} 2024-09-17T08:23:57.190 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'} 2024-09-17T08:23:57.190 INFO:teuthology.task.install:Using flavor: default 2024-09-17T08:23:57.196 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-17T08:23:57.196 INFO:teuthology.task.install:extra packages: [] 2024-09-17T08:23:57.196 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9', 'tag': None, 'wait_for_package': False} 2024-09-17T08:23:57.197 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-17T08:23:57.198 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9', 'tag': None, 'wait_for_package': False} 2024-09-17T08:23:57.198 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-17T08:23:57.342 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/ 2024-09-17T08:23:57.342 INFO:teuthology.task.install.rpm:Package version is 18.2.4-855.g19e2a2be 2024-09-17T08:23:57.347 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/ 2024-09-17T08:23:57.347 INFO:teuthology.task.install.rpm:Package version is 18.2.4-855.g19e2a2be 2024-09-17T08:23:57.450 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-17T08:23:57.451 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-17T08:23:57.451 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-17T08:23:57.459 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-17T08:23:57.459 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-17T08:23:57.459 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-17T08:23:57.482 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-17T08:23:57.482 DEBUG:teuthology.orchestra.run.smithi118:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-17T08:23:57.485 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-17T08:23:57.485 DEBUG:teuthology.orchestra.run.smithi170:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-17T08:23:57.550 DEBUG:teuthology.orchestra.run.smithi170:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-17T08:23:57.552 DEBUG:teuthology.orchestra.run.smithi118:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-17T08:23:57.626 DEBUG:teuthology.orchestra.run.smithi170:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-17T08:23:57.631 DEBUG:teuthology.orchestra.run.smithi118:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-17T08:23:57.690 INFO:teuthology.orchestra.run.smithi170.stdout:check_obsoletes = 1 2024-09-17T08:23:57.691 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum clean all 2024-09-17T08:23:57.697 INFO:teuthology.orchestra.run.smithi118.stdout:check_obsoletes = 1 2024-09-17T08:23:57.699 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum clean all 2024-09-17T08:23:57.927 INFO:teuthology.orchestra.run.smithi118.stdout:45 files removed 2024-09-17T08:23:57.928 INFO:teuthology.orchestra.run.smithi170.stdout:45 files removed 2024-09-17T08:23:57.955 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph-radosgw 2024-09-17T08:23:57.956 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum -y install ceph-radosgw 2024-09-17T08:23:58.508 INFO:teuthology.orchestra.run.smithi170.stdout:ceph packages for x86_64 293 kB/s | 77 kB 00:00 2024-09-17T08:23:58.521 INFO:teuthology.orchestra.run.smithi118.stdout:ceph packages for x86_64 279 kB/s | 77 kB 00:00 2024-09-17T08:23:58.783 INFO:teuthology.orchestra.run.smithi170.stdout:ceph noarch packages 49 kB/s | 11 kB 00:00 2024-09-17T08:23:58.822 INFO:teuthology.orchestra.run.smithi118.stdout:ceph noarch packages 44 kB/s | 11 kB 00:00 2024-09-17T08:23:59.042 INFO:teuthology.orchestra.run.smithi170.stdout:ceph source packages 8.4 kB/s | 1.9 kB 00:00 2024-09-17T08:23:59.214 INFO:teuthology.orchestra.run.smithi118.stdout:ceph source packages 5.3 kB/s | 1.9 kB 00:00 2024-09-17T08:24:00.809 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - BaseOS 4.8 MB/s | 8.3 MB 00:01 2024-09-17T08:24:02.122 INFO:teuthology.orchestra.run.smithi118.stdout:CentOS Stream 9 - BaseOS 2.9 MB/s | 8.3 MB 00:02 2024-09-17T08:24:02.752 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - AppStream 23 MB/s | 20 MB 00:00 2024-09-17T08:24:04.024 INFO:teuthology.orchestra.run.smithi118.stdout:CentOS Stream 9 - AppStream 25 MB/s | 20 MB 00:00 2024-09-17T08:24:07.944 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - CRB 4.7 MB/s | 6.5 MB 00:01 2024-09-17T08:24:09.378 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - Extras packages 87 kB/s | 19 kB 00:00 2024-09-17T08:24:09.533 INFO:teuthology.orchestra.run.smithi118.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.5 MB 00:01 2024-09-17T08:24:10.666 INFO:teuthology.orchestra.run.smithi170.stdout:Extra Packages for Enterprise Linux 19 MB/s | 23 MB 00:01 2024-09-17T08:24:11.051 INFO:teuthology.orchestra.run.smithi118.stdout:CentOS Stream 9 - Extras packages 68 kB/s | 19 kB 00:00 2024-09-17T08:24:11.805 INFO:teuthology.orchestra.run.smithi118.stdout:Extra Packages for Enterprise Linux 36 MB/s | 23 MB 00:00 2024-09-17T08:24:16.005 INFO:teuthology.orchestra.run.smithi170.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-17T08:24:17.244 INFO:teuthology.orchestra.run.smithi118.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-17T08:24:18.242 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-09-17T08:24:18.244 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:24:18.244 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2024-09-17T08:24:18.244 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:24:18.245 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2024-09-17T08:24:18.245 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-radosgw x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 7.7 M 2024-09-17T08:24:18.245 INFO:teuthology.orchestra.run.smithi170.stdout:Upgrading: 2024-09-17T08:24:18.245 INFO:teuthology.orchestra.run.smithi170.stdout: librados2 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 3.3 M 2024-09-17T08:24:18.245 INFO:teuthology.orchestra.run.smithi170.stdout: librbd1 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 3.0 M 2024-09-17T08:24:18.245 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2024-09-17T08:24:18.245 INFO:teuthology.orchestra.run.smithi170.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-17T08:24:18.245 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 5.1 M 2024-09-17T08:24:18.245 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-common x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 18 M 2024-09-17T08:24:18.245 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-selinux x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 25 k 2024-09-17T08:24:18.245 INFO:teuthology.orchestra.run.smithi170.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-17T08:24:18.245 INFO:teuthology.orchestra.run.smithi170.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-17T08:24:18.246 INFO:teuthology.orchestra.run.smithi170.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-17T08:24:18.246 INFO:teuthology.orchestra.run.smithi170.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-17T08:24:18.246 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs2 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 712 k 2024-09-17T08:24:18.246 INFO:teuthology.orchestra.run.smithi170.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-17T08:24:18.246 INFO:teuthology.orchestra.run.smithi170.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-17T08:24:18.246 INFO:teuthology.orchestra.run.smithi170.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-17T08:24:18.246 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 477 k 2024-09-17T08:24:18.246 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-17T08:24:18.246 INFO:teuthology.orchestra.run.smithi170.stdout: librgw2 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 4.5 M 2024-09-17T08:24:18.246 INFO:teuthology.orchestra.run.smithi170.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-17T08:24:18.246 INFO:teuthology.orchestra.run.smithi170.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-17T08:24:18.246 INFO:teuthology.orchestra.run.smithi170.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-17T08:24:18.246 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 45 k 2024-09-17T08:24:18.247 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 129 k 2024-09-17T08:24:18.247 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cephfs x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 162 k 2024-09-17T08:24:18.247 INFO:teuthology.orchestra.run.smithi170.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-17T08:24:18.247 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rados x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 321 k 2024-09-17T08:24:18.247 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rbd x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 300 k 2024-09-17T08:24:18.247 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rgw x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 99 k 2024-09-17T08:24:18.247 INFO:teuthology.orchestra.run.smithi170.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-17T08:24:18.247 INFO:teuthology.orchestra.run.smithi170.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-17T08:24:18.247 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:24:18.247 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2024-09-17T08:24:18.247 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:24:18.247 INFO:teuthology.orchestra.run.smithi170.stdout:Install 28 Packages 2024-09-17T08:24:18.247 INFO:teuthology.orchestra.run.smithi170.stdout:Upgrade 2 Packages 2024-09-17T08:24:18.248 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:24:18.248 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 53 M 2024-09-17T08:24:18.248 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2024-09-17T08:24:18.965 INFO:teuthology.orchestra.run.smithi170.stdout:(1/30): ceph-base-18.2.4-855.g19e2a2be.el9.x86_ 7.6 MB/s | 5.1 MB 00:00 2024-09-17T08:24:19.065 INFO:teuthology.orchestra.run.smithi170.stdout:(2/30): ceph-radosgw-18.2.4-855.g19e2a2be.el9.x 10 MB/s | 7.7 MB 00:00 2024-09-17T08:24:19.082 INFO:teuthology.orchestra.run.smithi170.stdout:(3/30): ceph-selinux-18.2.4-855.g19e2a2be.el9.x 215 kB/s | 25 kB 00:00 2024-09-17T08:24:19.116 INFO:teuthology.orchestra.run.smithi170.stdout:(4/30): libcephfs2-18.2.4-855.g19e2a2be.el9.x86 14 MB/s | 712 kB 00:00 2024-09-17T08:24:19.141 INFO:teuthology.orchestra.run.smithi170.stdout:(5/30): libradosstriper1-18.2.4-855.g19e2a2be.e 8.0 MB/s | 477 kB 00:00 2024-09-17T08:24:19.191 INFO:teuthology.orchestra.run.smithi170.stdout:(6/30): python3-ceph-argparse-18.2.4-855.g19e2a 898 kB/s | 45 kB 00:00 2024-09-17T08:24:19.267 INFO:teuthology.orchestra.run.smithi170.stdout:(7/30): python3-ceph-common-18.2.4-855.g19e2a2b 1.7 MB/s | 129 kB 00:00 2024-09-17T08:24:19.458 INFO:teuthology.orchestra.run.smithi170.stdout:(8/30): ceph-common-18.2.4-855.g19e2a2be.el9.x8 16 MB/s | 18 MB 00:01 2024-09-17T08:24:19.484 INFO:teuthology.orchestra.run.smithi170.stdout:(9/30): python3-cephfs-18.2.4-855.g19e2a2be.el9 746 kB/s | 162 kB 00:00 2024-09-17T08:24:19.497 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2024-09-17T08:24:19.499 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:24:19.499 INFO:teuthology.orchestra.run.smithi118.stdout: Package Arch Version Repository Size 2024-09-17T08:24:19.500 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:24:19.500 INFO:teuthology.orchestra.run.smithi118.stdout:Installing: 2024-09-17T08:24:19.500 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-radosgw x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 7.7 M 2024-09-17T08:24:19.500 INFO:teuthology.orchestra.run.smithi118.stdout:Upgrading: 2024-09-17T08:24:19.500 INFO:teuthology.orchestra.run.smithi118.stdout: librados2 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 3.3 M 2024-09-17T08:24:19.500 INFO:teuthology.orchestra.run.smithi118.stdout: librbd1 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 3.0 M 2024-09-17T08:24:19.500 INFO:teuthology.orchestra.run.smithi118.stdout:Installing dependencies: 2024-09-17T08:24:19.500 INFO:teuthology.orchestra.run.smithi118.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-17T08:24:19.500 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-base x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 5.1 M 2024-09-17T08:24:19.500 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-common x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 18 M 2024-09-17T08:24:19.500 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-selinux x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 25 k 2024-09-17T08:24:19.500 INFO:teuthology.orchestra.run.smithi118.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-17T08:24:19.500 INFO:teuthology.orchestra.run.smithi118.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-17T08:24:19.501 INFO:teuthology.orchestra.run.smithi118.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-17T08:24:19.501 INFO:teuthology.orchestra.run.smithi118.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-17T08:24:19.501 INFO:teuthology.orchestra.run.smithi118.stdout: libcephfs2 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 712 k 2024-09-17T08:24:19.501 INFO:teuthology.orchestra.run.smithi118.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-17T08:24:19.501 INFO:teuthology.orchestra.run.smithi118.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-17T08:24:19.501 INFO:teuthology.orchestra.run.smithi118.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-17T08:24:19.501 INFO:teuthology.orchestra.run.smithi118.stdout: libradosstriper1 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 477 k 2024-09-17T08:24:19.501 INFO:teuthology.orchestra.run.smithi118.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-17T08:24:19.501 INFO:teuthology.orchestra.run.smithi118.stdout: librgw2 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 4.5 M 2024-09-17T08:24:19.501 INFO:teuthology.orchestra.run.smithi118.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-17T08:24:19.501 INFO:teuthology.orchestra.run.smithi118.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-17T08:24:19.501 INFO:teuthology.orchestra.run.smithi118.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-17T08:24:19.501 INFO:teuthology.orchestra.run.smithi118.stdout: python3-ceph-argparse x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 45 k 2024-09-17T08:24:19.502 INFO:teuthology.orchestra.run.smithi118.stdout: python3-ceph-common x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 129 k 2024-09-17T08:24:19.502 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cephfs x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 162 k 2024-09-17T08:24:19.502 INFO:teuthology.orchestra.run.smithi118.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-17T08:24:19.502 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rados x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 321 k 2024-09-17T08:24:19.502 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rbd x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 300 k 2024-09-17T08:24:19.502 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rgw x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 99 k 2024-09-17T08:24:19.502 INFO:teuthology.orchestra.run.smithi118.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-17T08:24:19.502 INFO:teuthology.orchestra.run.smithi118.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-17T08:24:19.502 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:24:19.502 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction Summary 2024-09-17T08:24:19.502 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:24:19.502 INFO:teuthology.orchestra.run.smithi118.stdout:Install 28 Packages 2024-09-17T08:24:19.502 INFO:teuthology.orchestra.run.smithi118.stdout:Upgrade 2 Packages 2024-09-17T08:24:19.502 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:24:19.503 INFO:teuthology.orchestra.run.smithi118.stdout:Total download size: 53 M 2024-09-17T08:24:19.503 INFO:teuthology.orchestra.run.smithi118.stdout:Downloading Packages: 2024-09-17T08:24:19.517 INFO:teuthology.orchestra.run.smithi170.stdout:(10/30): python3-rados-18.2.4-855.g19e2a2be.el9 5.4 MB/s | 321 kB 00:00 2024-09-17T08:24:19.543 INFO:teuthology.orchestra.run.smithi170.stdout:(11/30): python3-rbd-18.2.4-855.g19e2a2be.el9.x 5.0 MB/s | 300 kB 00:00 2024-09-17T08:24:19.569 INFO:teuthology.orchestra.run.smithi170.stdout:(12/30): python3-rgw-18.2.4-855.g19e2a2be.el9.x 1.9 MB/s | 99 kB 00:00 2024-09-17T08:24:19.643 INFO:teuthology.orchestra.run.smithi170.stdout:(13/30): librgw2-18.2.4-855.g19e2a2be.el9.x86_6 8.5 MB/s | 4.5 MB 00:00 2024-09-17T08:24:19.752 INFO:teuthology.orchestra.run.smithi170.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 512 kB/s | 107 kB 00:00 2024-09-17T08:24:19.780 INFO:teuthology.orchestra.run.smithi170.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 332 kB/s | 45 kB 00:00 2024-09-17T08:24:19.805 INFO:teuthology.orchestra.run.smithi170.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 676 kB/s | 160 kB 00:00 2024-09-17T08:24:19.856 INFO:teuthology.orchestra.run.smithi170.stdout:(17/30): python3-prettytable-0.7.2-27.el9.noarc 847 kB/s | 42 kB 00:00 2024-09-17T08:24:19.914 INFO:teuthology.orchestra.run.smithi170.stdout:(18/30): fmt-8.1.1-5.el9.x86_64.rpm 1.9 MB/s | 111 kB 00:00 2024-09-17T08:24:19.940 INFO:teuthology.orchestra.run.smithi170.stdout:(19/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.8 MB/s | 292 kB 00:00 2024-09-17T08:24:19.965 INFO:teuthology.orchestra.run.smithi170.stdout:(20/30): librdkafka-1.6.1-102.el9.x86_64.rpm 3.0 MB/s | 662 kB 00:00 2024-09-17T08:24:20.007 INFO:teuthology.orchestra.run.smithi170.stdout:(21/30): gperftools-libs-2.9.1-3.el9.x86_64.rpm 3.2 MB/s | 308 kB 00:00 2024-09-17T08:24:20.024 INFO:teuthology.orchestra.run.smithi170.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 3.2 MB/s | 49 kB 00:00 2024-09-17T08:24:20.041 INFO:teuthology.orchestra.run.smithi170.stdout:(23/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 381 kB/s | 26 kB 00:00 2024-09-17T08:24:20.045 INFO:teuthology.orchestra.run.smithi118.stdout:(1/30): ceph-base-18.2.4-855.g19e2a2be.el9.x86_ 12 MB/s | 5.1 MB 00:00 2024-09-17T08:24:20.066 INFO:teuthology.orchestra.run.smithi170.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-09-17T08:24:20.096 INFO:teuthology.orchestra.run.smithi118.stdout:(2/30): ceph-selinux-18.2.4-855.g19e2a2be.el9.x 503 kB/s | 25 kB 00:00 2024-09-17T08:24:20.116 INFO:teuthology.orchestra.run.smithi170.stdout:(25/30): re2-20211101-20.el9.x86_64.rpm 3.8 MB/s | 191 kB 00:00 2024-09-17T08:24:20.183 INFO:teuthology.orchestra.run.smithi170.stdout:(26/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.8 MB/s | 839 kB 00:00 2024-09-17T08:24:20.246 INFO:teuthology.orchestra.run.smithi118.stdout:(3/30): ceph-radosgw-18.2.4-855.g19e2a2be.el9.x 12 MB/s | 7.7 MB 00:00 2024-09-17T08:24:20.258 INFO:teuthology.orchestra.run.smithi170.stdout:(27/30): libarrow-9.0.0-11.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-09-17T08:24:20.288 INFO:teuthology.orchestra.run.smithi118.stdout:(4/30): libcephfs2-18.2.4-855.g19e2a2be.el9.x86 3.6 MB/s | 712 kB 00:00 2024-09-17T08:24:20.314 INFO:teuthology.orchestra.run.smithi118.stdout:(5/30): libradosstriper1-18.2.4-855.g19e2a2be.e 7.0 MB/s | 477 kB 00:00 2024-09-17T08:24:20.342 INFO:teuthology.orchestra.run.smithi170.stdout:(28/30): thrift-0.15.0-2.el9.x86_64.rpm 7.0 MB/s | 1.6 MB 00:00 2024-09-17T08:24:20.389 INFO:teuthology.orchestra.run.smithi118.stdout:(6/30): python3-ceph-argparse-18.2.4-855.g19e2a 600 kB/s | 45 kB 00:00 2024-09-17T08:24:20.489 INFO:teuthology.orchestra.run.smithi118.stdout:(7/30): python3-ceph-common-18.2.4-855.g19e2a2b 1.3 MB/s | 129 kB 00:00 2024-09-17T08:24:20.559 INFO:teuthology.orchestra.run.smithi170.stdout:(29/30): librados2-18.2.4-855.g19e2a2be.el9.x86 8.8 MB/s | 3.3 MB 00:00 2024-09-17T08:24:20.573 INFO:teuthology.orchestra.run.smithi118.stdout:(8/30): python3-cephfs-18.2.4-855.g19e2a2be.el9 1.9 MB/s | 162 kB 00:00 2024-09-17T08:24:20.640 INFO:teuthology.orchestra.run.smithi118.stdout:(9/30): python3-rados-18.2.4-855.g19e2a2be.el9. 4.7 MB/s | 321 kB 00:00 2024-09-17T08:24:20.651 INFO:teuthology.orchestra.run.smithi170.stdout:(30/30): librbd1-18.2.4-855.g19e2a2be.el9.x86_6 7.7 MB/s | 3.0 MB 00:00 2024-09-17T08:24:20.655 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:24:20.655 INFO:teuthology.orchestra.run.smithi170.stdout:Total 22 MB/s | 53 MB 00:02 2024-09-17T08:24:20.724 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2024-09-17T08:24:20.758 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2024-09-17T08:24:20.758 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2024-09-17T08:24:20.865 INFO:teuthology.orchestra.run.smithi118.stdout:(10/30): ceph-common-18.2.4-855.g19e2a2be.el9.x 15 MB/s | 18 MB 00:01 2024-09-17T08:24:20.932 INFO:teuthology.orchestra.run.smithi118.stdout:(11/30): librgw2-18.2.4-855.g19e2a2be.el9.x86_6 6.9 MB/s | 4.5 MB 00:00 2024-09-17T08:24:20.957 INFO:teuthology.orchestra.run.smithi118.stdout:(12/30): python3-rgw-18.2.4-855.g19e2a2be.el9.x 1.1 MB/s | 99 kB 00:00 2024-09-17T08:24:20.982 INFO:teuthology.orchestra.run.smithi118.stdout:(13/30): python3-rbd-18.2.4-855.g19e2a2be.el9.x 876 kB/s | 300 kB 00:00 2024-09-17T08:24:21.041 INFO:teuthology.orchestra.run.smithi118.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 984 kB/s | 107 kB 00:00 2024-09-17T08:24:21.066 INFO:teuthology.orchestra.run.smithi118.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 541 kB/s | 45 kB 00:00 2024-09-17T08:24:21.101 INFO:teuthology.orchestra.run.smithi118.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.1 MB/s | 160 kB 00:00 2024-09-17T08:24:21.126 INFO:teuthology.orchestra.run.smithi118.stdout:(17/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 292 kB 00:00 2024-09-17T08:24:21.151 INFO:teuthology.orchestra.run.smithi118.stdout:(18/30): python3-prettytable-0.7.2-27.el9.noarc 841 kB/s | 42 kB 00:00 2024-09-17T08:24:21.185 INFO:teuthology.orchestra.run.smithi118.stdout:(19/30): librdkafka-1.6.1-102.el9.x86_64.rpm 4.5 MB/s | 662 kB 00:00 2024-09-17T08:24:21.216 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2024-09-17T08:24:21.216 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2024-09-17T08:24:21.260 INFO:teuthology.orchestra.run.smithi118.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 827 kB/s | 111 kB 00:00 2024-09-17T08:24:21.402 INFO:teuthology.orchestra.run.smithi118.stdout:(21/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 186 kB/s | 26 kB 00:00 2024-09-17T08:24:21.427 INFO:teuthology.orchestra.run.smithi118.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-17T08:24:21.461 INFO:teuthology.orchestra.run.smithi118.stdout:(23/30): gperftools-libs-2.9.1-3.el9.x86_64.rpm 994 kB/s | 308 kB 00:00 2024-09-17T08:24:21.486 INFO:teuthology.orchestra.run.smithi118.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-09-17T08:24:21.525 INFO:teuthology.orchestra.run.smithi118.stdout:(25/30): re2-20211101-20.el9.x86_64.rpm 4.9 MB/s | 191 kB 00:00 2024-09-17T08:24:21.558 INFO:teuthology.orchestra.run.smithi118.stdout:(26/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.5 MB/s | 839 kB 00:00 2024-09-17T08:24:21.617 INFO:teuthology.orchestra.run.smithi118.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-17T08:24:21.692 INFO:teuthology.orchestra.run.smithi118.stdout:(28/30): libarrow-9.0.0-11.el9.x86_64.rpm 9.0 MB/s | 4.4 MB 00:00 2024-09-17T08:24:21.801 INFO:teuthology.orchestra.run.smithi118.stdout:(29/30): librados2-18.2.4-855.g19e2a2be.el9.x86 14 MB/s | 3.3 MB 00:00 2024-09-17T08:24:21.876 INFO:teuthology.orchestra.run.smithi118.stdout:(30/30): librbd1-18.2.4-855.g19e2a2be.el9.x86_6 12 MB/s | 3.0 MB 00:00 2024-09-17T08:24:21.880 INFO:teuthology.orchestra.run.smithi118.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:24:21.881 INFO:teuthology.orchestra.run.smithi118.stdout:Total 22 MB/s | 53 MB 00:02 2024-09-17T08:24:21.949 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction check 2024-09-17T08:24:21.983 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction check succeeded. 2024-09-17T08:24:21.983 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction test 2024-09-17T08:24:22.433 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction test succeeded. 2024-09-17T08:24:22.433 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction 2024-09-17T08:24:22.547 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2024-09-17T08:24:22.647 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-09-17T08:24:22.689 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-09-17T08:24:22.748 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-09-17T08:24:23.159 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-09-17T08:24:23.222 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 5/32 2024-09-17T08:24:23.313 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 5/32 2024-09-17T08:24:23.406 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rados-2:18.2.4-855.g19e2a2be.el9.x86_64 6/32 2024-09-17T08:24:23.440 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 7/32 2024-09-17T08:24:23.502 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 7/32 2024-09-17T08:24:23.531 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-09-17T08:24:23.582 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ceph-argparse-2:18.2.4-855.g19e2a2be.el9.x 9/32 2024-09-17T08:24:23.663 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cephfs-2:18.2.4-855.g19e2a2be.el9.x86_64 10/32 2024-09-17T08:24:23.692 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libradosstriper1-2:18.2.4-855.g19e2a2be.el9.x86_64 11/32 2024-09-17T08:24:23.758 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libradosstriper1-2:18.2.4-855.g19e2a2be.el9.x86_64 11/32 2024-09-17T08:24:23.766 INFO:teuthology.orchestra.run.smithi118.stdout: Preparing : 1/1 2024-09-17T08:24:23.869 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-09-17T08:24:23.871 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/32 2024-09-17T08:24:23.906 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-09-17T08:24:23.908 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-09-17T08:24:23.949 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-09-17T08:24:23.960 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-09-17T08:24:23.990 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/32 2024-09-17T08:24:24.017 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-09-17T08:24:24.053 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-09-17T08:24:24.086 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-09-17T08:24:24.282 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-09-17T08:24:24.394 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-09-17T08:24:24.405 INFO:teuthology.orchestra.run.smithi118.stdout: Upgrading : librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 5/32 2024-09-17T08:24:24.432 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 20/32 2024-09-17T08:24:24.498 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 5/32 2024-09-17T08:24:24.522 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 20/32 2024-09-17T08:24:24.553 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rbd-2:18.2.4-855.g19e2a2be.el9.x86_64 21/32 2024-09-17T08:24:24.581 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-rados-2:18.2.4-855.g19e2a2be.el9.x86_64 6/32 2024-09-17T08:24:24.589 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-09-17T08:24:24.623 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 7/32 2024-09-17T08:24:24.686 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 7/32 2024-09-17T08:24:24.715 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-09-17T08:24:24.767 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-ceph-argparse-2:18.2.4-855.g19e2a2be.el9.x 9/32 2024-09-17T08:24:24.841 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-cephfs-2:18.2.4-855.g19e2a2be.el9.x86_64 10/32 2024-09-17T08:24:24.876 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libradosstriper1-2:18.2.4-855.g19e2a2be.el9.x86_64 11/32 2024-09-17T08:24:24.942 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: libradosstriper1-2:18.2.4-855.g19e2a2be.el9.x86_64 11/32 2024-09-17T08:24:25.003 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-09-17T08:24:25.034 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librgw2-2:18.2.4-855.g19e2a2be.el9.x86_64 24/32 2024-09-17T08:24:25.059 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/32 2024-09-17T08:24:25.093 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-09-17T08:24:25.106 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librgw2-2:18.2.4-855.g19e2a2be.el9.x86_64 24/32 2024-09-17T08:24:25.134 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-09-17T08:24:25.165 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/32 2024-09-17T08:24:25.183 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rgw-2:18.2.4-855.g19e2a2be.el9.x86_64 25/32 2024-09-17T08:24:25.193 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-09-17T08:24:25.212 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ceph-common-2:18.2.4-855.g19e2a2be.el9.x86 26/32 2024-09-17T08:24:25.229 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-09-17T08:24:25.275 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-09-17T08:24:25.580 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-09-17T08:24:25.624 INFO:teuthology.orchestra.run.smithi118.stdout: Upgrading : librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 20/32 2024-09-17T08:24:25.723 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 20/32 2024-09-17T08:24:25.755 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-rbd-2:18.2.4-855.g19e2a2be.el9.x86_64 21/32 2024-09-17T08:24:25.807 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-09-17T08:24:26.232 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-09-17T08:24:26.261 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : librgw2-2:18.2.4-855.g19e2a2be.el9.x86_64 24/32 2024-09-17T08:24:26.332 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: librgw2-2:18.2.4-855.g19e2a2be.el9.x86_64 24/32 2024-09-17T08:24:26.434 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-rgw-2:18.2.4-855.g19e2a2be.el9.x86_64 25/32 2024-09-17T08:24:26.463 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-ceph-common-2:18.2.4-855.g19e2a2be.el9.x86 26/32 2024-09-17T08:24:27.372 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 27/32 2024-09-17T08:24:27.417 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 27/32 2024-09-17T08:24:27.963 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 27/32 2024-09-17T08:24:28.004 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 28/32 2024-09-17T08:24:28.110 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 28/32 2024-09-17T08:24:28.110 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-17T08:24:28.110 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-17T08:24:28.110 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:24:28.139 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-selinux-2:18.2.4-855.g19e2a2be.el9.x86_64 29/32 2024-09-17T08:24:28.610 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 27/32 2024-09-17T08:24:28.644 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 27/32 2024-09-17T08:24:29.179 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 27/32 2024-09-17T08:24:29.214 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 28/32 2024-09-17T08:24:29.313 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 28/32 2024-09-17T08:24:29.313 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-17T08:24:29.313 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-17T08:24:29.313 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:24:29.357 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-selinux-2:18.2.4-855.g19e2a2be.el9.x86_64 29/32 2024-09-17T08:24:38.319 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-selinux-2:18.2.4-855.g19e2a2be.el9.x86_64 29/32 2024-09-17T08:24:38.320 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /sys 2024-09-17T08:24:38.320 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /proc 2024-09-17T08:24:38.320 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /mnt 2024-09-17T08:24:38.320 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /var/tmp 2024-09-17T08:24:38.320 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /home 2024-09-17T08:24:38.320 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /root 2024-09-17T08:24:38.320 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /tmp 2024-09-17T08:24:38.320 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:24:39.098 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-radosgw-2:18.2.4-855.g19e2a2be.el9.x86_64 30/32 2024-09-17T08:24:39.124 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-855.g19e2a2be.el9.x86_64 30/32 2024-09-17T08:24:39.124 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T08:24:39.124 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-17T08:24:39.124 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T08:24:39.124 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T08:24:39.124 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:24:39.126 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-17T08:24:39.204 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-17T08:24:39.205 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-17T08:24:39.795 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-selinux-2:18.2.4-855.g19e2a2be.el9.x86_64 29/32 2024-09-17T08:24:39.796 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /sys 2024-09-17T08:24:39.796 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /proc 2024-09-17T08:24:39.796 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /mnt 2024-09-17T08:24:39.796 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /var/tmp 2024-09-17T08:24:39.796 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /home 2024-09-17T08:24:39.796 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /root 2024-09-17T08:24:39.796 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /tmp 2024-09-17T08:24:39.796 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:24:40.408 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-17T08:24:40.408 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 1/32 2024-09-17T08:24:40.408 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 2/32 2024-09-17T08:24:40.408 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-radosgw-2:18.2.4-855.g19e2a2be.el9.x86_64 3/32 2024-09-17T08:24:40.408 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-selinux-2:18.2.4-855.g19e2a2be.el9.x86_64 4/32 2024-09-17T08:24:40.408 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 5/32 2024-09-17T08:24:40.408 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libradosstriper1-2:18.2.4-855.g19e2a2be.el9.x86_64 6/32 2024-09-17T08:24:40.408 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librgw2-2:18.2.4-855.g19e2a2be.el9.x86_64 7/32 2024-09-17T08:24:40.408 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ceph-argparse-2:18.2.4-855.g19e2a2be.el9.x 8/32 2024-09-17T08:24:40.408 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ceph-common-2:18.2.4-855.g19e2a2be.el9.x86 9/32 2024-09-17T08:24:40.409 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cephfs-2:18.2.4-855.g19e2a2be.el9.x86_64 10/32 2024-09-17T08:24:40.409 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rados-2:18.2.4-855.g19e2a2be.el9.x86_64 11/32 2024-09-17T08:24:40.409 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rbd-2:18.2.4-855.g19e2a2be.el9.x86_64 12/32 2024-09-17T08:24:40.409 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rgw-2:18.2.4-855.g19e2a2be.el9.x86_64 13/32 2024-09-17T08:24:40.409 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-09-17T08:24:40.409 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-09-17T08:24:40.409 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-09-17T08:24:40.409 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-09-17T08:24:40.409 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-09-17T08:24:40.409 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-09-17T08:24:40.409 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-09-17T08:24:40.409 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 21/32 2024-09-17T08:24:40.409 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-09-17T08:24:40.410 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-09-17T08:24:40.410 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-09-17T08:24:40.410 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-09-17T08:24:40.410 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-09-17T08:24:40.410 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : re2-1:20211101-20.el9.x86_64 27/32 2024-09-17T08:24:40.410 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-09-17T08:24:40.410 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 29/32 2024-09-17T08:24:40.410 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-09-17T08:24:40.410 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 31/32 2024-09-17T08:24:40.541 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-radosgw-2:18.2.4-855.g19e2a2be.el9.x86_64 30/32 2024-09-17T08:24:40.566 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-855.g19e2a2be.el9.x86_64 30/32 2024-09-17T08:24:40.566 INFO:teuthology.orchestra.run.smithi118.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T08:24:40.566 INFO:teuthology.orchestra.run.smithi118.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-17T08:24:40.566 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T08:24:40.567 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T08:24:40.567 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:24:40.568 INFO:teuthology.orchestra.run.smithi118.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-17T08:24:40.652 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-17T08:24:40.652 INFO:teuthology.orchestra.run.smithi118.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-17T08:24:41.461 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-09-17T08:24:41.461 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:24:41.462 INFO:teuthology.orchestra.run.smithi170.stdout:Upgraded: 2024-09-17T08:24:41.462 INFO:teuthology.orchestra.run.smithi170.stdout: librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:24:41.462 INFO:teuthology.orchestra.run.smithi170.stdout: librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:24:41.462 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2024-09-17T08:24:41.462 INFO:teuthology.orchestra.run.smithi170.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-17T08:24:41.462 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:24:41.462 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:24:41.463 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-radosgw-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:24:41.463 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-selinux-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:24:41.463 INFO:teuthology.orchestra.run.smithi170.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-17T08:24:41.463 INFO:teuthology.orchestra.run.smithi170.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-17T08:24:41.463 INFO:teuthology.orchestra.run.smithi170.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-17T08:24:41.464 INFO:teuthology.orchestra.run.smithi170.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-17T08:24:41.464 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:24:41.464 INFO:teuthology.orchestra.run.smithi170.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-17T08:24:41.464 INFO:teuthology.orchestra.run.smithi170.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-17T08:24:41.464 INFO:teuthology.orchestra.run.smithi170.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-17T08:24:41.464 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:24:41.464 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-17T08:24:41.465 INFO:teuthology.orchestra.run.smithi170.stdout: librgw2-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:24:41.465 INFO:teuthology.orchestra.run.smithi170.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-17T08:24:41.465 INFO:teuthology.orchestra.run.smithi170.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-17T08:24:41.465 INFO:teuthology.orchestra.run.smithi170.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-17T08:24:41.465 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:24:41.465 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:24:41.465 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cephfs-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:24:41.466 INFO:teuthology.orchestra.run.smithi170.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-17T08:24:41.466 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rados-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:24:41.466 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rbd-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:24:41.466 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rgw-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:24:41.466 INFO:teuthology.orchestra.run.smithi170.stdout: re2-1:20211101-20.el9.x86_64 2024-09-17T08:24:41.466 INFO:teuthology.orchestra.run.smithi170.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-17T08:24:41.466 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:24:41.467 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-09-17T08:24:41.831 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph-test 2024-09-17T08:24:41.845 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-17T08:24:41.845 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 1/32 2024-09-17T08:24:41.845 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 2/32 2024-09-17T08:24:41.845 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-radosgw-2:18.2.4-855.g19e2a2be.el9.x86_64 3/32 2024-09-17T08:24:41.845 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-selinux-2:18.2.4-855.g19e2a2be.el9.x86_64 4/32 2024-09-17T08:24:41.845 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 5/32 2024-09-17T08:24:41.845 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libradosstriper1-2:18.2.4-855.g19e2a2be.el9.x86_64 6/32 2024-09-17T08:24:41.845 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librgw2-2:18.2.4-855.g19e2a2be.el9.x86_64 7/32 2024-09-17T08:24:41.845 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-ceph-argparse-2:18.2.4-855.g19e2a2be.el9.x 8/32 2024-09-17T08:24:41.845 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-ceph-common-2:18.2.4-855.g19e2a2be.el9.x86 9/32 2024-09-17T08:24:41.846 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-cephfs-2:18.2.4-855.g19e2a2be.el9.x86_64 10/32 2024-09-17T08:24:41.846 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-rados-2:18.2.4-855.g19e2a2be.el9.x86_64 11/32 2024-09-17T08:24:41.846 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-rbd-2:18.2.4-855.g19e2a2be.el9.x86_64 12/32 2024-09-17T08:24:41.846 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-rgw-2:18.2.4-855.g19e2a2be.el9.x86_64 13/32 2024-09-17T08:24:41.846 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-09-17T08:24:41.846 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-09-17T08:24:41.846 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-09-17T08:24:41.846 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-09-17T08:24:41.846 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-09-17T08:24:41.846 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-09-17T08:24:41.846 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-09-17T08:24:41.846 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 21/32 2024-09-17T08:24:41.847 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-09-17T08:24:41.847 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-09-17T08:24:41.847 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-09-17T08:24:41.847 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-09-17T08:24:41.847 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-09-17T08:24:41.847 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : re2-1:20211101-20.el9.x86_64 27/32 2024-09-17T08:24:41.847 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-09-17T08:24:41.847 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 29/32 2024-09-17T08:24:41.847 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-09-17T08:24:41.847 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 31/32 2024-09-17T08:24:42.406 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:00:27 ago on Tue 17 Sep 2024 08:24:15 AM UTC. 2024-09-17T08:24:42.543 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-09-17T08:24:42.544 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:24:42.544 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2024-09-17T08:24:42.544 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:24:42.544 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2024-09-17T08:24:42.544 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-test x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 40 M 2024-09-17T08:24:42.544 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2024-09-17T08:24:42.544 INFO:teuthology.orchestra.run.smithi170.stdout: libcephsqlite x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 166 k 2024-09-17T08:24:42.544 INFO:teuthology.orchestra.run.smithi170.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-17T08:24:42.544 INFO:teuthology.orchestra.run.smithi170.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-17T08:24:42.545 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:24:42.545 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2024-09-17T08:24:42.545 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:24:42.545 INFO:teuthology.orchestra.run.smithi170.stdout:Install 4 Packages 2024-09-17T08:24:42.545 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:24:42.545 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 40 M 2024-09-17T08:24:42.545 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 164 M 2024-09-17T08:24:42.546 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2024-09-17T08:24:42.728 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-09-17T08:24:42.728 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:24:42.728 INFO:teuthology.orchestra.run.smithi118.stdout:Upgraded: 2024-09-17T08:24:42.728 INFO:teuthology.orchestra.run.smithi118.stdout: librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:24:42.728 INFO:teuthology.orchestra.run.smithi118.stdout: librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:24:42.728 INFO:teuthology.orchestra.run.smithi118.stdout:Installed: 2024-09-17T08:24:42.729 INFO:teuthology.orchestra.run.smithi118.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-17T08:24:42.729 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:24:42.729 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:24:42.729 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-radosgw-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:24:42.729 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-selinux-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:24:42.729 INFO:teuthology.orchestra.run.smithi118.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-17T08:24:42.729 INFO:teuthology.orchestra.run.smithi118.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-17T08:24:42.729 INFO:teuthology.orchestra.run.smithi118.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-17T08:24:42.729 INFO:teuthology.orchestra.run.smithi118.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-17T08:24:42.729 INFO:teuthology.orchestra.run.smithi118.stdout: libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:24:42.729 INFO:teuthology.orchestra.run.smithi118.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-17T08:24:42.729 INFO:teuthology.orchestra.run.smithi118.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-17T08:24:42.730 INFO:teuthology.orchestra.run.smithi118.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-17T08:24:42.730 INFO:teuthology.orchestra.run.smithi118.stdout: libradosstriper1-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:24:42.730 INFO:teuthology.orchestra.run.smithi118.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-17T08:24:42.730 INFO:teuthology.orchestra.run.smithi118.stdout: librgw2-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:24:42.730 INFO:teuthology.orchestra.run.smithi118.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-17T08:24:42.730 INFO:teuthology.orchestra.run.smithi118.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-17T08:24:42.730 INFO:teuthology.orchestra.run.smithi118.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-17T08:24:42.730 INFO:teuthology.orchestra.run.smithi118.stdout: python3-ceph-argparse-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:24:42.730 INFO:teuthology.orchestra.run.smithi118.stdout: python3-ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:24:42.730 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cephfs-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:24:42.730 INFO:teuthology.orchestra.run.smithi118.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-17T08:24:42.730 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rados-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:24:42.730 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rbd-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:24:42.730 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rgw-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:24:42.731 INFO:teuthology.orchestra.run.smithi118.stdout: re2-1:20211101-20.el9.x86_64 2024-09-17T08:24:42.731 INFO:teuthology.orchestra.run.smithi118.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-17T08:24:42.731 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:24:42.731 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2024-09-17T08:24:42.809 INFO:teuthology.orchestra.run.smithi170.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.4 MB/s | 304 kB 00:00 2024-09-17T08:24:42.842 INFO:teuthology.orchestra.run.smithi170.stdout:(2/4): libcephsqlite-18.2.4-855.g19e2a2be.el9.x 673 kB/s | 166 kB 00:00 2024-09-17T08:24:42.867 INFO:teuthology.orchestra.run.smithi170.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-17T08:24:43.122 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum -y install ceph-test 2024-09-17T08:24:43.692 INFO:teuthology.orchestra.run.smithi118.stdout:Last metadata expiration check: 0:00:26 ago on Tue 17 Sep 2024 08:24:17 AM UTC. 2024-09-17T08:24:43.822 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2024-09-17T08:24:43.823 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:24:43.823 INFO:teuthology.orchestra.run.smithi118.stdout: Package Arch Version Repository Size 2024-09-17T08:24:43.823 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:24:43.823 INFO:teuthology.orchestra.run.smithi118.stdout:Installing: 2024-09-17T08:24:43.823 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-test x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 40 M 2024-09-17T08:24:43.823 INFO:teuthology.orchestra.run.smithi118.stdout:Installing dependencies: 2024-09-17T08:24:43.823 INFO:teuthology.orchestra.run.smithi118.stdout: libcephsqlite x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 166 k 2024-09-17T08:24:43.823 INFO:teuthology.orchestra.run.smithi118.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-17T08:24:43.823 INFO:teuthology.orchestra.run.smithi118.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-17T08:24:43.823 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:24:43.824 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction Summary 2024-09-17T08:24:43.824 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:24:43.824 INFO:teuthology.orchestra.run.smithi118.stdout:Install 4 Packages 2024-09-17T08:24:43.824 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:24:43.824 INFO:teuthology.orchestra.run.smithi118.stdout:Total download size: 40 M 2024-09-17T08:24:43.824 INFO:teuthology.orchestra.run.smithi118.stdout:Installed size: 164 M 2024-09-17T08:24:43.825 INFO:teuthology.orchestra.run.smithi118.stdout:Downloading Packages: 2024-09-17T08:24:43.909 INFO:teuthology.orchestra.run.smithi170.stdout:(4/4): ceph-test-18.2.4-855.g19e2a2be.el9.x86_6 30 MB/s | 40 MB 00:01 2024-09-17T08:24:43.912 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:24:43.912 INFO:teuthology.orchestra.run.smithi170.stdout:Total 29 MB/s | 40 MB 00:01 2024-09-17T08:24:43.941 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2024-09-17T08:24:43.960 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2024-09-17T08:24:43.961 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2024-09-17T08:24:44.104 INFO:teuthology.orchestra.run.smithi118.stdout:(1/4): libcephsqlite-18.2.4-855.g19e2a2be.el9.x 732 kB/s | 166 kB 00:00 2024-09-17T08:24:44.287 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2024-09-17T08:24:44.287 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2024-09-17T08:24:44.679 INFO:teuthology.orchestra.run.smithi118.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 379 kB/s | 304 kB 00:00 2024-09-17T08:24:44.737 INFO:teuthology.orchestra.run.smithi118.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 102 kB/s | 64 kB 00:00 2024-09-17T08:24:44.769 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2024-09-17T08:24:44.859 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-17T08:24:44.901 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-17T08:24:44.928 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephsqlite-2:18.2.4-855.g19e2a2be.el9.x86_64 3/4 2024-09-17T08:24:45.271 INFO:teuthology.orchestra.run.smithi118.stdout:(4/4): ceph-test-18.2.4-855.g19e2a2be.el9.x86_6 28 MB/s | 40 MB 00:01 2024-09-17T08:24:45.273 INFO:teuthology.orchestra.run.smithi118.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:24:45.273 INFO:teuthology.orchestra.run.smithi118.stdout:Total 28 MB/s | 40 MB 00:01 2024-09-17T08:24:45.300 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction check 2024-09-17T08:24:45.318 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction check succeeded. 2024-09-17T08:24:45.318 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction test 2024-09-17T08:24:45.646 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction test succeeded. 2024-09-17T08:24:45.646 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction 2024-09-17T08:24:46.125 INFO:teuthology.orchestra.run.smithi118.stdout: Preparing : 1/1 2024-09-17T08:24:46.211 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-17T08:24:46.263 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-17T08:24:46.289 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libcephsqlite-2:18.2.4-855.g19e2a2be.el9.x86_64 3/4 2024-09-17T08:24:48.621 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libcephsqlite-2:18.2.4-855.g19e2a2be.el9.x86_64 3/4 2024-09-17T08:24:48.695 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-test-2:18.2.4-855.g19e2a2be.el9.x86_64 4/4 2024-09-17T08:24:50.002 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: libcephsqlite-2:18.2.4-855.g19e2a2be.el9.x86_64 3/4 2024-09-17T08:24:50.072 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-test-2:18.2.4-855.g19e2a2be.el9.x86_64 4/4 2024-09-17T08:24:50.853 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-test-2:18.2.4-855.g19e2a2be.el9.x86_64 4/4 2024-09-17T08:24:50.853 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-test-2:18.2.4-855.g19e2a2be.el9.x86_64 1/4 2024-09-17T08:24:50.853 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephsqlite-2:18.2.4-855.g19e2a2be.el9.x86_64 2/4 2024-09-17T08:24:50.853 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-17T08:24:51.143 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-17T08:24:51.144 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:24:51.144 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2024-09-17T08:24:51.144 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-test-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:24:51.144 INFO:teuthology.orchestra.run.smithi170.stdout: libcephsqlite-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:24:51.144 INFO:teuthology.orchestra.run.smithi170.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-17T08:24:51.144 INFO:teuthology.orchestra.run.smithi170.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-17T08:24:51.144 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:24:51.144 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-09-17T08:24:51.442 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph 2024-09-17T08:24:52.028 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:00:37 ago on Tue 17 Sep 2024 08:24:15 AM UTC. 2024-09-17T08:24:52.164 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-09-17T08:24:52.166 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:24:52.166 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2024-09-17T08:24:52.166 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:24:52.167 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2024-09-17T08:24:52.167 INFO:teuthology.orchestra.run.smithi170.stdout: ceph x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 6.5 k 2024-09-17T08:24:52.167 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2024-09-17T08:24:52.167 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mds x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 2.1 M 2024-09-17T08:24:52.167 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 1.5 M 2024-09-17T08:24:52.167 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 246 k 2024-09-17T08:24:52.167 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mon x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 4.7 M 2024-09-17T08:24:52.167 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-osd x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 17 M 2024-09-17T08:24:52.167 INFO:teuthology.orchestra.run.smithi170.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-17T08:24:52.167 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-17T08:24:52.167 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-17T08:24:52.167 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-17T08:24:52.168 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-17T08:24:52.168 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-17T08:24:52.168 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-17T08:24:52.168 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-17T08:24:52.168 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-17T08:24:52.168 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-17T08:24:52.168 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-17T08:24:52.168 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-17T08:24:52.168 INFO:teuthology.orchestra.run.smithi170.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-17T08:24:52.168 INFO:teuthology.orchestra.run.smithi170.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-17T08:24:52.168 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-17T08:24:52.168 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-17T08:24:52.168 INFO:teuthology.orchestra.run.smithi170.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-17T08:24:52.169 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-17T08:24:52.169 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-17T08:24:52.169 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-17T08:24:52.169 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-17T08:24:52.169 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-17T08:24:52.169 INFO:teuthology.orchestra.run.smithi170.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-17T08:24:52.169 INFO:teuthology.orchestra.run.smithi170.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-17T08:24:52.169 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-17T08:24:52.169 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-17T08:24:52.169 INFO:teuthology.orchestra.run.smithi170.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-17T08:24:52.169 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:24:52.169 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2024-09-17T08:24:52.169 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:24:52.170 INFO:teuthology.orchestra.run.smithi170.stdout:Install 33 Packages 2024-09-17T08:24:52.170 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:24:52.170 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 29 M 2024-09-17T08:24:52.170 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 104 M 2024-09-17T08:24:52.170 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2024-09-17T08:24:52.278 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-test-2:18.2.4-855.g19e2a2be.el9.x86_64 4/4 2024-09-17T08:24:52.278 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-test-2:18.2.4-855.g19e2a2be.el9.x86_64 1/4 2024-09-17T08:24:52.278 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libcephsqlite-2:18.2.4-855.g19e2a2be.el9.x86_64 2/4 2024-09-17T08:24:52.278 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-17T08:24:52.515 INFO:teuthology.orchestra.run.smithi170.stdout:(1/33): ceph-18.2.4-855.g19e2a2be.el9.x86_64.rp 51 kB/s | 6.5 kB 00:00 2024-09-17T08:24:52.598 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-17T08:24:52.598 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:24:52.598 INFO:teuthology.orchestra.run.smithi118.stdout:Installed: 2024-09-17T08:24:52.598 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-test-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:24:52.599 INFO:teuthology.orchestra.run.smithi118.stdout: libcephsqlite-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:24:52.599 INFO:teuthology.orchestra.run.smithi118.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-17T08:24:52.599 INFO:teuthology.orchestra.run.smithi118.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-17T08:24:52.599 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:24:52.599 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2024-09-17T08:24:52.699 INFO:teuthology.orchestra.run.smithi170.stdout:(2/33): ceph-mgr-18.2.4-855.g19e2a2be.el9.x86_6 4.7 MB/s | 1.5 MB 00:00 2024-09-17T08:24:52.740 INFO:teuthology.orchestra.run.smithi170.stdout:(3/33): ceph-mds-18.2.4-855.g19e2a2be.el9.x86_6 5.9 MB/s | 2.1 MB 00:00 2024-09-17T08:24:52.824 INFO:teuthology.orchestra.run.smithi170.stdout:(4/33): ceph-mgr-modules-core-18.2.4-855.g19e2a 2.9 MB/s | 246 kB 00:00 2024-09-17T08:24:52.972 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum -y install ceph 2024-09-17T08:24:53.024 INFO:teuthology.orchestra.run.smithi170.stdout:(5/33): ceph-mon-18.2.4-855.g19e2a2be.el9.x86_6 9.2 MB/s | 4.7 MB 00:00 2024-09-17T08:24:53.241 INFO:teuthology.orchestra.run.smithi170.stdout:(6/33): ceph-osd-18.2.4-855.g19e2a2be.el9.x86_6 30 MB/s | 17 MB 00:00 2024-09-17T08:24:53.500 INFO:teuthology.orchestra.run.smithi170.stdout:(7/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 375 kB/s | 253 kB 00:00 2024-09-17T08:24:53.554 INFO:teuthology.orchestra.run.smithi118.stdout:Last metadata expiration check: 0:00:36 ago on Tue 17 Sep 2024 08:24:17 AM UTC. 2024-09-17T08:24:53.625 INFO:teuthology.orchestra.run.smithi170.stdout:(8/33): python3-pycparser-2.20-6.el9.noarch.rpm 1.1 MB/s | 135 kB 00:00 2024-09-17T08:24:53.700 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2024-09-17T08:24:53.703 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:24:53.703 INFO:teuthology.orchestra.run.smithi118.stdout: Package Arch Version Repository Size 2024-09-17T08:24:53.703 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:24:53.703 INFO:teuthology.orchestra.run.smithi118.stdout:Installing: 2024-09-17T08:24:53.703 INFO:teuthology.orchestra.run.smithi118.stdout: ceph x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 6.5 k 2024-09-17T08:24:53.703 INFO:teuthology.orchestra.run.smithi118.stdout:Installing dependencies: 2024-09-17T08:24:53.703 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mds x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 2.1 M 2024-09-17T08:24:53.703 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 1.5 M 2024-09-17T08:24:53.703 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-modules-core noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 246 k 2024-09-17T08:24:53.703 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mon x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 4.7 M 2024-09-17T08:24:53.703 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-osd x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 17 M 2024-09-17T08:24:53.704 INFO:teuthology.orchestra.run.smithi118.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-17T08:24:53.704 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-17T08:24:53.704 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-17T08:24:53.704 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-17T08:24:53.704 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-17T08:24:53.704 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-17T08:24:53.704 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-17T08:24:53.704 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-17T08:24:53.704 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-17T08:24:53.704 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-17T08:24:53.704 INFO:teuthology.orchestra.run.smithi118.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-17T08:24:53.704 INFO:teuthology.orchestra.run.smithi118.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-17T08:24:53.704 INFO:teuthology.orchestra.run.smithi118.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-17T08:24:53.704 INFO:teuthology.orchestra.run.smithi118.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-17T08:24:53.705 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-17T08:24:53.705 INFO:teuthology.orchestra.run.smithi118.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-17T08:24:53.705 INFO:teuthology.orchestra.run.smithi118.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-17T08:24:53.705 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-17T08:24:53.705 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-17T08:24:53.705 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-17T08:24:53.705 INFO:teuthology.orchestra.run.smithi118.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-17T08:24:53.705 INFO:teuthology.orchestra.run.smithi118.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-17T08:24:53.705 INFO:teuthology.orchestra.run.smithi118.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-17T08:24:53.705 INFO:teuthology.orchestra.run.smithi118.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-17T08:24:53.705 INFO:teuthology.orchestra.run.smithi118.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-17T08:24:53.705 INFO:teuthology.orchestra.run.smithi118.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-17T08:24:53.705 INFO:teuthology.orchestra.run.smithi118.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-17T08:24:53.705 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:24:53.706 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction Summary 2024-09-17T08:24:53.706 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:24:53.706 INFO:teuthology.orchestra.run.smithi118.stdout:Install 33 Packages 2024-09-17T08:24:53.706 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:24:53.707 INFO:teuthology.orchestra.run.smithi118.stdout:Total download size: 29 M 2024-09-17T08:24:53.707 INFO:teuthology.orchestra.run.smithi118.stdout:Installed size: 104 M 2024-09-17T08:24:53.707 INFO:teuthology.orchestra.run.smithi118.stdout:Downloading Packages: 2024-09-17T08:24:53.717 INFO:teuthology.orchestra.run.smithi170.stdout:(9/33): python3-ply-3.11-14.el9.noarch.rpm 224 kB/s | 106 kB 00:00 2024-09-17T08:24:53.742 INFO:teuthology.orchestra.run.smithi170.stdout:(10/33): python3-requests-2.25.1-8.el9.noarch.r 1.0 MB/s | 125 kB 00:00 2024-09-17T08:24:53.934 INFO:teuthology.orchestra.run.smithi170.stdout:(11/33): python3-urllib3-1.26.5-6.el9.noarch.rp 989 kB/s | 215 kB 00:00 2024-09-17T08:24:54.039 INFO:teuthology.orchestra.run.smithi118.stdout:(1/33): ceph-18.2.4-855.g19e2a2be.el9.x86_64.rp 54 kB/s | 6.5 kB 00:00 2024-09-17T08:24:54.093 INFO:teuthology.orchestra.run.smithi170.stdout:(12/33): python3-markupsafe-1.1.1-12.el9.x86_64 219 kB/s | 35 kB 00:00 2024-09-17T08:24:54.159 INFO:teuthology.orchestra.run.smithi170.stdout:(13/33): python3-mako-1.1.4-6.el9.noarch.rpm 413 kB/s | 172 kB 00:00 2024-09-17T08:24:54.210 INFO:teuthology.orchestra.run.smithi170.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 437 kB/s | 51 kB 00:00 2024-09-17T08:24:54.235 INFO:teuthology.orchestra.run.smithi170.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 553 kB/s | 42 kB 00:00 2024-09-17T08:24:54.239 INFO:teuthology.orchestra.run.smithi118.stdout:(2/33): ceph-mds-18.2.4-855.g19e2a2be.el9.x86_6 6.5 MB/s | 2.1 MB 00:00 2024-09-17T08:24:54.281 INFO:teuthology.orchestra.run.smithi118.stdout:(3/33): ceph-mgr-18.2.4-855.g19e2a2be.el9.x86_6 4.0 MB/s | 1.5 MB 00:00 2024-09-17T08:24:54.285 INFO:teuthology.orchestra.run.smithi170.stdout:(16/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 576 kB/s | 43 kB 00:00 2024-09-17T08:24:54.353 INFO:teuthology.orchestra.run.smithi170.stdout:(17/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-17T08:24:54.357 INFO:teuthology.orchestra.run.smithi118.stdout:(4/33): ceph-mgr-modules-core-18.2.4-855.g19e2a 3.2 MB/s | 246 kB 00:00 2024-09-17T08:24:54.395 INFO:teuthology.orchestra.run.smithi170.stdout:(18/33): python3-cryptography-36.0.1-4.el9.x86_ 916 kB/s | 1.2 MB 00:01 2024-09-17T08:24:54.420 INFO:teuthology.orchestra.run.smithi170.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 159 kB/s | 11 kB 00:00 2024-09-17T08:24:54.462 INFO:teuthology.orchestra.run.smithi170.stdout:(20/33): python3-cherrypy-18.6.1-2.el9.noarch.r 2.0 MB/s | 358 kB 00:00 2024-09-17T08:24:54.473 INFO:teuthology.orchestra.run.smithi118.stdout:(5/33): ceph-mon-18.2.4-855.g19e2a2be.el9.x86_6 11 MB/s | 4.7 MB 00:00 2024-09-17T08:24:54.487 INFO:teuthology.orchestra.run.smithi170.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 387 kB/s | 23 kB 00:00 2024-09-17T08:24:54.513 INFO:teuthology.orchestra.run.smithi170.stdout:(22/33): python3-jaraco-classes-3.2.1-5.el9.noa 150 kB/s | 18 kB 00:00 2024-09-17T08:24:54.538 INFO:teuthology.orchestra.run.smithi170.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 264 kB/s | 19 kB 00:00 2024-09-17T08:24:54.563 INFO:teuthology.orchestra.run.smithi170.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 261 kB/s | 20 kB 00:00 2024-09-17T08:24:54.588 INFO:teuthology.orchestra.run.smithi170.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 923 kB/s | 46 kB 00:00 2024-09-17T08:24:54.607 INFO:teuthology.orchestra.run.smithi118.stdout:(6/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.0 MB/s | 253 kB 00:00 2024-09-17T08:24:54.614 INFO:teuthology.orchestra.run.smithi170.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-17T08:24:54.639 INFO:teuthology.orchestra.run.smithi170.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 326 kB/s | 16 kB 00:00 2024-09-17T08:24:54.665 INFO:teuthology.orchestra.run.smithi170.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-09-17T08:24:54.690 INFO:teuthology.orchestra.run.smithi170.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 712 kB/s | 36 kB 00:00 2024-09-17T08:24:54.715 INFO:teuthology.orchestra.run.smithi170.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-17T08:24:54.740 INFO:teuthology.orchestra.run.smithi170.stdout:(31/33): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-17T08:24:54.765 INFO:teuthology.orchestra.run.smithi170.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 399 kB/s | 20 kB 00:00 2024-09-17T08:24:54.766 INFO:teuthology.orchestra.run.smithi118.stdout:(7/33): ceph-osd-18.2.4-855.g19e2a2be.el9.x86_6 31 MB/s | 17 MB 00:00 2024-09-17T08:24:54.790 INFO:teuthology.orchestra.run.smithi170.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.1 MB/s | 427 kB 00:00 2024-09-17T08:24:54.791 INFO:teuthology.orchestra.run.smithi118.stdout:(8/33): python3-ply-3.11-14.el9.noarch.rpm 580 kB/s | 106 kB 00:00 2024-09-17T08:24:54.795 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:24:54.795 INFO:teuthology.orchestra.run.smithi170.stdout:Total 11 MB/s | 29 MB 00:02 2024-09-17T08:24:54.841 INFO:teuthology.orchestra.run.smithi118.stdout:(9/33): python3-requests-2.25.1-8.el9.noarch.rp 2.4 MB/s | 125 kB 00:00 2024-09-17T08:24:54.919 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2024-09-17T08:24:54.925 INFO:teuthology.orchestra.run.smithi118.stdout:(10/33): python3-urllib3-1.26.5-6.el9.noarch.rp 2.5 MB/s | 215 kB 00:00 2024-09-17T08:24:54.950 INFO:teuthology.orchestra.run.smithi118.stdout:(11/33): python3-pycparser-2.20-6.el9.noarch.rp 734 kB/s | 135 kB 00:00 2024-09-17T08:24:54.957 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2024-09-17T08:24:54.957 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2024-09-17T08:24:54.984 INFO:teuthology.orchestra.run.smithi118.stdout:(12/33): python3-cryptography-36.0.1-4.el9.x86_ 2.4 MB/s | 1.2 MB 00:00 2024-09-17T08:24:55.059 INFO:teuthology.orchestra.run.smithi118.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 320 kB/s | 35 kB 00:00 2024-09-17T08:24:55.118 INFO:teuthology.orchestra.run.smithi118.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 381 kB/s | 51 kB 00:00 2024-09-17T08:24:55.136 INFO:teuthology.orchestra.run.smithi118.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 552 kB/s | 42 kB 00:00 2024-09-17T08:24:55.160 INFO:teuthology.orchestra.run.smithi118.stdout:(16/33): python3-mako-1.1.4-6.el9.noarch.rpm 733 kB/s | 172 kB 00:00 2024-09-17T08:24:55.202 INFO:teuthology.orchestra.run.smithi118.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 518 kB/s | 43 kB 00:00 2024-09-17T08:24:55.228 INFO:teuthology.orchestra.run.smithi118.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 460 kB/s | 11 kB 00:00 2024-09-17T08:24:55.253 INFO:teuthology.orchestra.run.smithi118.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-17T08:24:55.278 INFO:teuthology.orchestra.run.smithi118.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 350 kB/s | 18 kB 00:00 2024-09-17T08:24:55.304 INFO:teuthology.orchestra.run.smithi118.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 461 kB/s | 23 kB 00:00 2024-09-17T08:24:55.329 INFO:teuthology.orchestra.run.smithi118.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-09-17T08:24:55.354 INFO:teuthology.orchestra.run.smithi118.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-09-17T08:24:55.359 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2024-09-17T08:24:55.359 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2024-09-17T08:24:55.379 INFO:teuthology.orchestra.run.smithi118.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 919 kB/s | 46 kB 00:00 2024-09-17T08:24:55.413 INFO:teuthology.orchestra.run.smithi118.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 1.3 MB/s | 79 kB 00:00 2024-09-17T08:24:55.438 INFO:teuthology.orchestra.run.smithi118.stdout:(26/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-09-17T08:24:55.463 INFO:teuthology.orchestra.run.smithi118.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 326 kB/s | 16 kB 00:00 2024-09-17T08:24:55.489 INFO:teuthology.orchestra.run.smithi118.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-17T08:24:55.514 INFO:teuthology.orchestra.run.smithi118.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 711 kB/s | 36 kB 00:00 2024-09-17T08:24:55.540 INFO:teuthology.orchestra.run.smithi118.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-09-17T08:24:55.565 INFO:teuthology.orchestra.run.smithi118.stdout:(31/33): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-17T08:24:55.590 INFO:teuthology.orchestra.run.smithi118.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 399 kB/s | 20 kB 00:00 2024-09-17T08:24:55.615 INFO:teuthology.orchestra.run.smithi118.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.1 MB/s | 427 kB 00:00 2024-09-17T08:24:55.619 INFO:teuthology.orchestra.run.smithi118.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:24:55.619 INFO:teuthology.orchestra.run.smithi118.stdout:Total 15 MB/s | 29 MB 00:01 2024-09-17T08:24:55.737 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction check 2024-09-17T08:24:55.775 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction check succeeded. 2024-09-17T08:24:55.775 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction test 2024-09-17T08:24:56.179 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction test succeeded. 2024-09-17T08:24:56.179 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction 2024-09-17T08:24:56.572 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2024-09-17T08:24:56.635 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-09-17T08:24:56.666 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-09-17T08:24:56.699 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-09-17T08:24:56.798 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-09-17T08:24:56.877 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-09-17T08:24:56.919 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/33 2024-09-17T08:24:56.962 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-09-17T08:24:56.993 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-09-17T08:24:57.027 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-09-17T08:24:57.061 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-09-17T08:24:57.101 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-09-17T08:24:57.136 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-09-17T08:24:57.171 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-09-17T08:24:57.209 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-09-17T08:24:57.289 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-09-17T08:24:57.426 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-09-17T08:24:57.450 INFO:teuthology.orchestra.run.smithi118.stdout: Preparing : 1/1 2024-09-17T08:24:57.507 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-09-17T08:24:57.527 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-09-17T08:24:57.559 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-09-17T08:24:57.569 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 18/33 2024-09-17T08:24:57.600 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-09-17T08:24:57.625 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-09-17T08:24:57.674 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-09-17T08:24:57.694 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-09-17T08:24:57.729 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-09-17T08:24:57.779 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-09-17T08:24:57.828 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/33 2024-09-17T08:24:57.871 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-09-17T08:24:57.901 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-09-17T08:24:57.912 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-09-17T08:24:57.936 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-09-17T08:24:57.969 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-09-17T08:24:57.971 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-09-17T08:24:58.010 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-09-17T08:24:58.053 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-09-17T08:24:58.057 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-09-17T08:24:58.088 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-09-17T08:24:58.126 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-09-17T08:24:58.154 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-09-17T08:24:58.200 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-09-17T08:24:58.206 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-09-17T08:24:58.320 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-09-17T08:24:58.341 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-09-17T08:24:58.427 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-09-17T08:24:58.488 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-855.g19e2a2be.el9.n 28/33 2024-09-17T08:24:58.505 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 18/33 2024-09-17T08:24:58.516 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 29/33 2024-09-17T08:24:58.543 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 29/33 2024-09-17T08:24:58.543 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T08:24:58.543 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-17T08:24:58.543 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T08:24:58.543 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T08:24:58.543 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:24:58.544 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-09-17T08:24:58.601 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-09-17T08:24:58.665 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-09-17T08:24:58.853 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-09-17T08:24:58.906 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-09-17T08:24:58.985 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-09-17T08:24:59.074 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-09-17T08:24:59.114 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-09-17T08:24:59.230 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-09-17T08:24:59.392 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-855.g19e2a2be.el9.n 28/33 2024-09-17T08:24:59.424 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 29/33 2024-09-17T08:24:59.450 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 29/33 2024-09-17T08:24:59.450 INFO:teuthology.orchestra.run.smithi118.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T08:24:59.450 INFO:teuthology.orchestra.run.smithi118.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-17T08:24:59.450 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T08:24:59.450 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T08:24:59.450 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:24:59.976 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-osd-2:18.2.4-855.g19e2a2be.el9.x86_64 30/33 2024-09-17T08:25:00.006 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-osd-2:18.2.4-855.g19e2a2be.el9.x86_64 30/33 2024-09-17T08:25:00.006 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T08:25:00.006 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-17T08:25:00.006 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T08:25:00.006 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T08:25:00.006 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:00.477 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mon-2:18.2.4-855.g19e2a2be.el9.x86_64 31/33 2024-09-17T08:25:00.502 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mon-2:18.2.4-855.g19e2a2be.el9.x86_64 31/33 2024-09-17T08:25:00.503 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T08:25:00.503 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-17T08:25:00.503 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T08:25:00.503 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T08:25:00.503 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:00.702 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mds-2:18.2.4-855.g19e2a2be.el9.x86_64 32/33 2024-09-17T08:25:00.727 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mds-2:18.2.4-855.g19e2a2be.el9.x86_64 32/33 2024-09-17T08:25:00.727 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T08:25:00.727 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-17T08:25:00.727 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T08:25:00.727 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T08:25:00.728 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:00.791 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-2:18.2.4-855.g19e2a2be.el9.x86_64 33/33 2024-09-17T08:25:00.934 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-osd-2:18.2.4-855.g19e2a2be.el9.x86_64 30/33 2024-09-17T08:25:00.970 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-osd-2:18.2.4-855.g19e2a2be.el9.x86_64 30/33 2024-09-17T08:25:00.970 INFO:teuthology.orchestra.run.smithi118.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T08:25:00.970 INFO:teuthology.orchestra.run.smithi118.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-17T08:25:00.970 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T08:25:00.970 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T08:25:00.971 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:01.435 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mon-2:18.2.4-855.g19e2a2be.el9.x86_64 31/33 2024-09-17T08:25:01.459 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-mon-2:18.2.4-855.g19e2a2be.el9.x86_64 31/33 2024-09-17T08:25:01.459 INFO:teuthology.orchestra.run.smithi118.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T08:25:01.459 INFO:teuthology.orchestra.run.smithi118.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-17T08:25:01.459 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T08:25:01.460 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T08:25:01.460 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:01.706 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mds-2:18.2.4-855.g19e2a2be.el9.x86_64 32/33 2024-09-17T08:25:01.730 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-mds-2:18.2.4-855.g19e2a2be.el9.x86_64 32/33 2024-09-17T08:25:01.731 INFO:teuthology.orchestra.run.smithi118.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T08:25:01.731 INFO:teuthology.orchestra.run.smithi118.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-17T08:25:01.731 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T08:25:01.731 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T08:25:01.731 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:01.799 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-2:18.2.4-855.g19e2a2be.el9.x86_64 33/33 2024-09-17T08:25:02.650 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-2:18.2.4-855.g19e2a2be.el9.x86_64 33/33 2024-09-17T08:25:02.650 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-2:18.2.4-855.g19e2a2be.el9.x86_64 1/33 2024-09-17T08:25:02.650 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mds-2:18.2.4-855.g19e2a2be.el9.x86_64 2/33 2024-09-17T08:25:02.650 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 3/33 2024-09-17T08:25:02.650 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mon-2:18.2.4-855.g19e2a2be.el9.x86_64 4/33 2024-09-17T08:25:02.650 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-osd-2:18.2.4-855.g19e2a2be.el9.x86_64 5/33 2024-09-17T08:25:02.650 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-855.g19e2a2be.el9.n 6/33 2024-09-17T08:25:02.650 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-09-17T08:25:02.650 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-09-17T08:25:02.650 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-09-17T08:25:02.650 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-09-17T08:25:02.651 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-09-17T08:25:02.651 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/33 2024-09-17T08:25:02.651 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-09-17T08:25:02.651 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-09-17T08:25:02.651 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-09-17T08:25:02.651 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-09-17T08:25:02.651 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-09-17T08:25:02.651 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-09-17T08:25:02.651 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-09-17T08:25:02.651 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-09-17T08:25:02.651 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-09-17T08:25:02.651 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-09-17T08:25:02.651 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-09-17T08:25:02.652 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-09-17T08:25:02.652 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-09-17T08:25:02.652 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-09-17T08:25:02.652 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-09-17T08:25:02.652 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-09-17T08:25:02.653 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-09-17T08:25:02.653 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-09-17T08:25:02.653 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 31/33 2024-09-17T08:25:02.653 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-09-17T08:25:03.529 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-09-17T08:25:03.529 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:03.529 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2024-09-17T08:25:03.529 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:25:03.529 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mds-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:25:03.529 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:25:03.529 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-17T08:25:03.529 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mon-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:25:03.530 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-osd-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:25:03.530 INFO:teuthology.orchestra.run.smithi170.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-17T08:25:03.530 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-17T08:25:03.530 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-17T08:25:03.530 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-17T08:25:03.530 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-17T08:25:03.530 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-17T08:25:03.530 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-17T08:25:03.530 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-17T08:25:03.530 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-17T08:25:03.530 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-17T08:25:03.531 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-17T08:25:03.531 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-17T08:25:03.531 INFO:teuthology.orchestra.run.smithi170.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-17T08:25:03.531 INFO:teuthology.orchestra.run.smithi170.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-17T08:25:03.531 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-17T08:25:03.531 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ply-3.11-14.el9.noarch 2024-09-17T08:25:03.531 INFO:teuthology.orchestra.run.smithi170.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-17T08:25:03.531 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-17T08:25:03.531 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-17T08:25:03.531 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-17T08:25:03.531 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-17T08:25:03.532 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-17T08:25:03.532 INFO:teuthology.orchestra.run.smithi170.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-17T08:25:03.532 INFO:teuthology.orchestra.run.smithi170.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-17T08:25:03.532 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-17T08:25:03.532 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-17T08:25:03.532 INFO:teuthology.orchestra.run.smithi170.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-17T08:25:03.532 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:03.532 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-09-17T08:25:03.686 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-2:18.2.4-855.g19e2a2be.el9.x86_64 33/33 2024-09-17T08:25:03.686 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-2:18.2.4-855.g19e2a2be.el9.x86_64 1/33 2024-09-17T08:25:03.686 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mds-2:18.2.4-855.g19e2a2be.el9.x86_64 2/33 2024-09-17T08:25:03.686 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 3/33 2024-09-17T08:25:03.686 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mon-2:18.2.4-855.g19e2a2be.el9.x86_64 4/33 2024-09-17T08:25:03.686 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-osd-2:18.2.4-855.g19e2a2be.el9.x86_64 5/33 2024-09-17T08:25:03.686 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-855.g19e2a2be.el9.n 6/33 2024-09-17T08:25:03.687 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-09-17T08:25:03.687 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-09-17T08:25:03.687 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-09-17T08:25:03.687 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-09-17T08:25:03.687 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-09-17T08:25:03.687 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/33 2024-09-17T08:25:03.687 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-09-17T08:25:03.687 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-09-17T08:25:03.687 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-09-17T08:25:03.687 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-09-17T08:25:03.687 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-09-17T08:25:03.687 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-09-17T08:25:03.687 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-09-17T08:25:03.687 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-09-17T08:25:03.688 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-09-17T08:25:03.688 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-09-17T08:25:03.688 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-09-17T08:25:03.688 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-09-17T08:25:03.688 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-09-17T08:25:03.688 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-09-17T08:25:03.688 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-09-17T08:25:03.688 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-09-17T08:25:03.689 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-09-17T08:25:03.689 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-09-17T08:25:03.689 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 31/33 2024-09-17T08:25:03.689 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-09-17T08:25:03.963 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph-base 2024-09-17T08:25:04.553 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-09-17T08:25:04.553 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:04.553 INFO:teuthology.orchestra.run.smithi118.stdout:Installed: 2024-09-17T08:25:04.553 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:25:04.553 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mds-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:25:04.553 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:25:04.553 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-modules-core-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-17T08:25:04.553 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mon-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:25:04.553 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-osd-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:25:04.553 INFO:teuthology.orchestra.run.smithi118.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-17T08:25:04.553 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-17T08:25:04.554 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-17T08:25:04.554 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-17T08:25:04.554 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-17T08:25:04.554 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-17T08:25:04.554 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-17T08:25:04.554 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-17T08:25:04.554 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-17T08:25:04.554 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-17T08:25:04.554 INFO:teuthology.orchestra.run.smithi118.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-17T08:25:04.554 INFO:teuthology.orchestra.run.smithi118.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-17T08:25:04.554 INFO:teuthology.orchestra.run.smithi118.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-17T08:25:04.554 INFO:teuthology.orchestra.run.smithi118.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-17T08:25:04.555 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-17T08:25:04.555 INFO:teuthology.orchestra.run.smithi118.stdout: python3-ply-3.11-14.el9.noarch 2024-09-17T08:25:04.555 INFO:teuthology.orchestra.run.smithi118.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-17T08:25:04.555 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-17T08:25:04.555 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-17T08:25:04.555 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-17T08:25:04.555 INFO:teuthology.orchestra.run.smithi118.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-17T08:25:04.555 INFO:teuthology.orchestra.run.smithi118.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-17T08:25:04.555 INFO:teuthology.orchestra.run.smithi118.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-17T08:25:04.555 INFO:teuthology.orchestra.run.smithi118.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-17T08:25:04.555 INFO:teuthology.orchestra.run.smithi118.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-17T08:25:04.555 INFO:teuthology.orchestra.run.smithi118.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-17T08:25:04.555 INFO:teuthology.orchestra.run.smithi118.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-17T08:25:04.555 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:04.556 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2024-09-17T08:25:04.556 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:00:49 ago on Tue 17 Sep 2024 08:24:15 AM UTC. 2024-09-17T08:25:04.645 INFO:teuthology.orchestra.run.smithi170.stdout:Package ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-17T08:25:04.687 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-09-17T08:25:04.688 INFO:teuthology.orchestra.run.smithi170.stdout:Nothing to do. 2024-09-17T08:25:04.688 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-09-17T08:25:04.764 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install cephadm 2024-09-17T08:25:04.962 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum -y install ceph-base 2024-09-17T08:25:05.336 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:00:50 ago on Tue 17 Sep 2024 08:24:15 AM UTC. 2024-09-17T08:25:05.466 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-09-17T08:25:05.467 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:25:05.467 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2024-09-17T08:25:05.467 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:25:05.467 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2024-09-17T08:25:05.467 INFO:teuthology.orchestra.run.smithi170.stdout: cephadm noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 226 k 2024-09-17T08:25:05.467 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:05.468 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2024-09-17T08:25:05.468 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:25:05.468 INFO:teuthology.orchestra.run.smithi170.stdout:Install 1 Package 2024-09-17T08:25:05.468 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:05.468 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 226 k 2024-09-17T08:25:05.468 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 217 k 2024-09-17T08:25:05.468 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2024-09-17T08:25:05.566 INFO:teuthology.orchestra.run.smithi118.stdout:Last metadata expiration check: 0:00:48 ago on Tue 17 Sep 2024 08:24:17 AM UTC. 2024-09-17T08:25:05.654 INFO:teuthology.orchestra.run.smithi118.stdout:Package ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-17T08:25:05.696 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2024-09-17T08:25:05.697 INFO:teuthology.orchestra.run.smithi118.stdout:Nothing to do. 2024-09-17T08:25:05.697 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2024-09-17T08:25:05.702 INFO:teuthology.orchestra.run.smithi170.stdout:cephadm-18.2.4-855.g19e2a2be.el9.noarch.rpm 968 kB/s | 226 kB 00:00 2024-09-17T08:25:05.702 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:25:05.703 INFO:teuthology.orchestra.run.smithi170.stdout:Total 961 kB/s | 226 kB 00:00 2024-09-17T08:25:05.703 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2024-09-17T08:25:05.708 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2024-09-17T08:25:05.708 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2024-09-17T08:25:05.711 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2024-09-17T08:25:05.711 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2024-09-17T08:25:05.777 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2024-09-17T08:25:05.780 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum -y install cephadm 2024-09-17T08:25:06.224 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/1 2024-09-17T08:25:06.314 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/1 2024-09-17T08:25:06.358 INFO:teuthology.orchestra.run.smithi118.stdout:Last metadata expiration check: 0:00:49 ago on Tue 17 Sep 2024 08:24:17 AM UTC. 2024-09-17T08:25:06.489 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2024-09-17T08:25:06.490 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:25:06.490 INFO:teuthology.orchestra.run.smithi118.stdout: Package Arch Version Repository Size 2024-09-17T08:25:06.490 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:25:06.490 INFO:teuthology.orchestra.run.smithi118.stdout:Installing: 2024-09-17T08:25:06.490 INFO:teuthology.orchestra.run.smithi118.stdout: cephadm noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 226 k 2024-09-17T08:25:06.490 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:06.490 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction Summary 2024-09-17T08:25:06.490 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:25:06.490 INFO:teuthology.orchestra.run.smithi118.stdout:Install 1 Package 2024-09-17T08:25:06.490 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:06.490 INFO:teuthology.orchestra.run.smithi118.stdout:Total download size: 226 k 2024-09-17T08:25:06.490 INFO:teuthology.orchestra.run.smithi118.stdout:Installed size: 217 k 2024-09-17T08:25:06.490 INFO:teuthology.orchestra.run.smithi118.stdout:Downloading Packages: 2024-09-17T08:25:06.655 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/1 2024-09-17T08:25:06.701 INFO:teuthology.orchestra.run.smithi118.stdout:cephadm-18.2.4-855.g19e2a2be.el9.noarch.rpm 1.0 MB/s | 226 kB 00:00 2024-09-17T08:25:06.702 INFO:teuthology.orchestra.run.smithi118.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:25:06.702 INFO:teuthology.orchestra.run.smithi118.stdout:Total 1.0 MB/s | 226 kB 00:00 2024-09-17T08:25:06.702 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction check 2024-09-17T08:25:06.708 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction check succeeded. 2024-09-17T08:25:06.708 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction test 2024-09-17T08:25:06.710 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction test succeeded. 2024-09-17T08:25:06.711 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction 2024-09-17T08:25:06.783 INFO:teuthology.orchestra.run.smithi118.stdout: Preparing : 1/1 2024-09-17T08:25:06.904 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/1 2024-09-17T08:25:06.905 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:06.905 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2024-09-17T08:25:06.905 INFO:teuthology.orchestra.run.smithi170.stdout: cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-17T08:25:06.905 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:06.905 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-09-17T08:25:07.130 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph-immutable-object-cache 2024-09-17T08:25:07.240 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/1 2024-09-17T08:25:07.322 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/1 2024-09-17T08:25:07.699 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/1 2024-09-17T08:25:07.714 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:00:52 ago on Tue 17 Sep 2024 08:24:15 AM UTC. 2024-09-17T08:25:07.847 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-09-17T08:25:07.847 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:25:07.848 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repo Size 2024-09-17T08:25:07.848 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:25:07.848 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2024-09-17T08:25:07.848 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 143 k 2024-09-17T08:25:07.848 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:07.848 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2024-09-17T08:25:07.848 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:25:07.848 INFO:teuthology.orchestra.run.smithi170.stdout:Install 1 Package 2024-09-17T08:25:07.848 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:07.848 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 143 k 2024-09-17T08:25:07.848 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 395 k 2024-09-17T08:25:07.848 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2024-09-17T08:25:07.944 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/1 2024-09-17T08:25:07.945 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:07.945 INFO:teuthology.orchestra.run.smithi118.stdout:Installed: 2024-09-17T08:25:07.945 INFO:teuthology.orchestra.run.smithi118.stdout: cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-17T08:25:07.945 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:07.945 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2024-09-17T08:25:08.051 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-immutable-object-cache-18.2.4-855.g19e2a2b 705 kB/s | 143 kB 00:00 2024-09-17T08:25:08.052 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:25:08.052 INFO:teuthology.orchestra.run.smithi170.stdout:Total 700 kB/s | 143 kB 00:00 2024-09-17T08:25:08.052 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2024-09-17T08:25:08.059 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2024-09-17T08:25:08.059 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2024-09-17T08:25:08.117 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2024-09-17T08:25:08.117 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2024-09-17T08:25:08.180 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum -y install ceph-immutable-object-cache 2024-09-17T08:25:08.274 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2024-09-17T08:25:08.361 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-855.g19e2a2be.e 1/1 2024-09-17T08:25:08.386 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-855.g19e2a2be.e 1/1 2024-09-17T08:25:08.386 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T08:25:08.386 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-17T08:25:08.386 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:08.775 INFO:teuthology.orchestra.run.smithi118.stdout:Last metadata expiration check: 0:00:51 ago on Tue 17 Sep 2024 08:24:17 AM UTC. 2024-09-17T08:25:08.911 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2024-09-17T08:25:08.912 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:25:08.912 INFO:teuthology.orchestra.run.smithi118.stdout: Package Arch Version Repo Size 2024-09-17T08:25:08.912 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:25:08.912 INFO:teuthology.orchestra.run.smithi118.stdout:Installing: 2024-09-17T08:25:08.912 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 143 k 2024-09-17T08:25:08.912 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:08.912 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction Summary 2024-09-17T08:25:08.912 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:25:08.912 INFO:teuthology.orchestra.run.smithi118.stdout:Install 1 Package 2024-09-17T08:25:08.913 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:08.913 INFO:teuthology.orchestra.run.smithi118.stdout:Total download size: 143 k 2024-09-17T08:25:08.913 INFO:teuthology.orchestra.run.smithi118.stdout:Installed size: 395 k 2024-09-17T08:25:08.913 INFO:teuthology.orchestra.run.smithi118.stdout:Downloading Packages: 2024-09-17T08:25:09.134 INFO:teuthology.orchestra.run.smithi118.stdout:ceph-immutable-object-cache-18.2.4-855.g19e2a2b 647 kB/s | 143 kB 00:00 2024-09-17T08:25:09.134 INFO:teuthology.orchestra.run.smithi118.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:25:09.135 INFO:teuthology.orchestra.run.smithi118.stdout:Total 642 kB/s | 143 kB 00:00 2024-09-17T08:25:09.135 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction check 2024-09-17T08:25:09.142 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction check succeeded. 2024-09-17T08:25:09.142 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction test 2024-09-17T08:25:09.200 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction test succeeded. 2024-09-17T08:25:09.200 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction 2024-09-17T08:25:09.285 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-855.g19e2a2be.e 1/1 2024-09-17T08:25:09.285 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:09.285 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2024-09-17T08:25:09.285 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-immutable-object-cache-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:25:09.285 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:09.285 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-09-17T08:25:09.351 INFO:teuthology.orchestra.run.smithi118.stdout: Preparing : 1/1 2024-09-17T08:25:09.418 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-855.g19e2a2be.e 1/1 2024-09-17T08:25:09.443 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-855.g19e2a2be.e 1/1 2024-09-17T08:25:09.443 INFO:teuthology.orchestra.run.smithi118.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T08:25:09.443 INFO:teuthology.orchestra.run.smithi118.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-17T08:25:09.443 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:09.479 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph-mgr 2024-09-17T08:25:10.069 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:00:55 ago on Tue 17 Sep 2024 08:24:15 AM UTC. 2024-09-17T08:25:10.160 INFO:teuthology.orchestra.run.smithi170.stdout:Package ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-17T08:25:10.205 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-09-17T08:25:10.206 INFO:teuthology.orchestra.run.smithi170.stdout:Nothing to do. 2024-09-17T08:25:10.206 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-09-17T08:25:10.276 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph-mgr-dashboard 2024-09-17T08:25:10.359 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-855.g19e2a2be.e 1/1 2024-09-17T08:25:10.359 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:10.359 INFO:teuthology.orchestra.run.smithi118.stdout:Installed: 2024-09-17T08:25:10.359 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-immutable-object-cache-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:25:10.359 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:10.359 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2024-09-17T08:25:10.578 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum -y install ceph-mgr 2024-09-17T08:25:10.855 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:00:55 ago on Tue 17 Sep 2024 08:24:15 AM UTC. 2024-09-17T08:25:11.002 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-09-17T08:25:11.002 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:25:11.002 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2024-09-17T08:25:11.002 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:25:11.002 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2024-09-17T08:25:11.002 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 3.5 M 2024-09-17T08:25:11.002 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2024-09-17T08:25:11.002 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-grafana-dashboards noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 24 k 2024-09-17T08:25:11.003 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-prometheus-alerts noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 16 k 2024-09-17T08:25:11.003 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-17T08:25:11.003 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-17T08:25:11.003 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:11.003 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2024-09-17T08:25:11.003 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:25:11.003 INFO:teuthology.orchestra.run.smithi170.stdout:Install 5 Packages 2024-09-17T08:25:11.003 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:11.003 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 3.8 M 2024-09-17T08:25:11.003 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 87 M 2024-09-17T08:25:11.003 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2024-09-17T08:25:11.179 INFO:teuthology.orchestra.run.smithi118.stdout:Last metadata expiration check: 0:00:54 ago on Tue 17 Sep 2024 08:24:17 AM UTC. 2024-09-17T08:25:11.272 INFO:teuthology.orchestra.run.smithi118.stdout:Package ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-17T08:25:11.316 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2024-09-17T08:25:11.317 INFO:teuthology.orchestra.run.smithi118.stdout:Nothing to do. 2024-09-17T08:25:11.317 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2024-09-17T08:25:11.340 INFO:teuthology.orchestra.run.smithi170.stdout:(1/5): ceph-grafana-dashboards-18.2.4-855.g19e2 170 kB/s | 24 kB 00:00 2024-09-17T08:25:11.365 INFO:teuthology.orchestra.run.smithi170.stdout:(2/5): ceph-prometheus-alerts-18.2.4-855.g19e2a 97 kB/s | 16 kB 00:00 2024-09-17T08:25:11.391 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum -y install ceph-mgr-dashboard 2024-09-17T08:25:11.408 INFO:teuthology.orchestra.run.smithi170.stdout:(3/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 460 kB/s | 31 kB 00:00 2024-09-17T08:25:11.432 INFO:teuthology.orchestra.run.smithi170.stdout:(4/5): python3-routes-2.5.1-5.el9.noarch.rpm 2.8 MB/s | 188 kB 00:00 2024-09-17T08:25:11.599 INFO:teuthology.orchestra.run.smithi170.stdout:(5/5): ceph-mgr-dashboard-18.2.4-855.g19e2a2be. 8.8 MB/s | 3.5 MB 00:00 2024-09-17T08:25:11.603 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:25:11.603 INFO:teuthology.orchestra.run.smithi170.stdout:Total 6.3 MB/s | 3.8 MB 00:00 2024-09-17T08:25:11.604 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2024-09-17T08:25:11.611 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2024-09-17T08:25:11.611 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2024-09-17T08:25:11.695 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2024-09-17T08:25:11.696 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2024-09-17T08:25:11.959 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2024-09-17T08:25:11.962 INFO:teuthology.orchestra.run.smithi118.stdout:Last metadata expiration check: 0:00:54 ago on Tue 17 Sep 2024 08:24:17 AM UTC. 2024-09-17T08:25:12.047 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-09-17T08:25:12.087 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-09-17T08:25:12.094 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2024-09-17T08:25:12.094 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:25:12.094 INFO:teuthology.orchestra.run.smithi118.stdout: Package Arch Version Repository Size 2024-09-17T08:25:12.095 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:25:12.095 INFO:teuthology.orchestra.run.smithi118.stdout:Installing: 2024-09-17T08:25:12.095 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-dashboard noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 3.5 M 2024-09-17T08:25:12.095 INFO:teuthology.orchestra.run.smithi118.stdout:Installing dependencies: 2024-09-17T08:25:12.095 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-grafana-dashboards noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 24 k 2024-09-17T08:25:12.095 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-prometheus-alerts noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 16 k 2024-09-17T08:25:12.095 INFO:teuthology.orchestra.run.smithi118.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-17T08:25:12.095 INFO:teuthology.orchestra.run.smithi118.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-17T08:25:12.095 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:12.095 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction Summary 2024-09-17T08:25:12.095 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:25:12.095 INFO:teuthology.orchestra.run.smithi118.stdout:Install 5 Packages 2024-09-17T08:25:12.095 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:12.096 INFO:teuthology.orchestra.run.smithi118.stdout:Total download size: 3.8 M 2024-09-17T08:25:12.096 INFO:teuthology.orchestra.run.smithi118.stdout:Installed size: 87 M 2024-09-17T08:25:12.096 INFO:teuthology.orchestra.run.smithi118.stdout:Downloading Packages: 2024-09-17T08:25:12.116 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-prometheus-alerts-2:18.2.4-855.g19e2a2be.el9.no 3/5 2024-09-17T08:25:12.289 INFO:teuthology.orchestra.run.smithi118.stdout:(1/5): ceph-grafana-dashboards-18.2.4-855.g19e2 162 kB/s | 24 kB 00:00 2024-09-17T08:25:12.314 INFO:teuthology.orchestra.run.smithi118.stdout:(2/5): ceph-prometheus-alerts-18.2.4-855.g19e2a 94 kB/s | 16 kB 00:00 2024-09-17T08:25:12.356 INFO:teuthology.orchestra.run.smithi118.stdout:(3/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 460 kB/s | 31 kB 00:00 2024-09-17T08:25:12.381 INFO:teuthology.orchestra.run.smithi118.stdout:(4/5): python3-routes-2.5.1-5.el9.noarch.rpm 2.8 MB/s | 188 kB 00:00 2024-09-17T08:25:12.548 INFO:teuthology.orchestra.run.smithi118.stdout:(5/5): ceph-mgr-dashboard-18.2.4-855.g19e2a2be. 8.6 MB/s | 3.5 MB 00:00 2024-09-17T08:25:12.551 INFO:teuthology.orchestra.run.smithi118.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:25:12.552 INFO:teuthology.orchestra.run.smithi118.stdout:Total 8.3 MB/s | 3.8 MB 00:00 2024-09-17T08:25:12.552 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction check 2024-09-17T08:25:12.559 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction check succeeded. 2024-09-17T08:25:12.559 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction test 2024-09-17T08:25:12.660 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction test succeeded. 2024-09-17T08:25:12.660 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction 2024-09-17T08:25:12.913 INFO:teuthology.orchestra.run.smithi118.stdout: Preparing : 1/1 2024-09-17T08:25:13.004 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-09-17T08:25:13.036 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-09-17T08:25:13.065 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-prometheus-alerts-2:18.2.4-855.g19e2a2be.el9.no 3/5 2024-09-17T08:25:13.161 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-grafana-dashboards-2:18.2.4-855.g19e2a2be.el9.n 4/5 2024-09-17T08:25:13.203 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-dashboard-2:18.2.4-855.g19e2a2be.el9.noarch 5/5 2024-09-17T08:25:14.118 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-grafana-dashboards-2:18.2.4-855.g19e2a2be.el9.n 4/5 2024-09-17T08:25:14.169 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mgr-dashboard-2:18.2.4-855.g19e2a2be.el9.noarch 5/5 2024-09-17T08:25:14.326 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.4-855.g19e2a2be.el9.noarch 5/5 2024-09-17T08:25:14.326 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-grafana-dashboards-2:18.2.4-855.g19e2a2be.el9.n 1/5 2024-09-17T08:25:14.326 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-dashboard-2:18.2.4-855.g19e2a2be.el9.noarch 2/5 2024-09-17T08:25:14.326 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-prometheus-alerts-2:18.2.4-855.g19e2a2be.el9.no 3/5 2024-09-17T08:25:14.326 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-09-17T08:25:14.593 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-09-17T08:25:14.593 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:14.593 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2024-09-17T08:25:14.593 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-grafana-dashboards-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-17T08:25:14.594 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-17T08:25:14.594 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-prometheus-alerts-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-17T08:25:14.594 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-17T08:25:14.594 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-17T08:25:14.594 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:14.594 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-09-17T08:25:14.817 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-17T08:25:15.266 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.4-855.g19e2a2be.el9.noarch 5/5 2024-09-17T08:25:15.266 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-grafana-dashboards-2:18.2.4-855.g19e2a2be.el9.n 1/5 2024-09-17T08:25:15.267 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mgr-dashboard-2:18.2.4-855.g19e2a2be.el9.noarch 2/5 2024-09-17T08:25:15.267 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-prometheus-alerts-2:18.2.4-855.g19e2a2be.el9.no 3/5 2024-09-17T08:25:15.267 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-09-17T08:25:15.386 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:01:00 ago on Tue 17 Sep 2024 08:24:15 AM UTC. 2024-09-17T08:25:15.517 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-09-17T08:25:15.518 INFO:teuthology.orchestra.run.smithi170.stdout:====================================================================================== 2024-09-17T08:25:15.518 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2024-09-17T08:25:15.518 INFO:teuthology.orchestra.run.smithi170.stdout:====================================================================================== 2024-09-17T08:25:15.518 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2024-09-17T08:25:15.518 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 7.4 M 2024-09-17T08:25:15.518 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2024-09-17T08:25:15.519 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-17T08:25:15.519 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-17T08:25:15.519 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-17T08:25:15.519 INFO:teuthology.orchestra.run.smithi170.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-17T08:25:15.519 INFO:teuthology.orchestra.run.smithi170.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-17T08:25:15.519 INFO:teuthology.orchestra.run.smithi170.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-17T08:25:15.519 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-17T08:25:15.519 INFO:teuthology.orchestra.run.smithi170.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-17T08:25:15.519 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-17T08:25:15.519 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-17T08:25:15.519 INFO:teuthology.orchestra.run.smithi170.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-17T08:25:15.519 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:15.519 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2024-09-17T08:25:15.520 INFO:teuthology.orchestra.run.smithi170.stdout:====================================================================================== 2024-09-17T08:25:15.520 INFO:teuthology.orchestra.run.smithi170.stdout:Install 12 Packages 2024-09-17T08:25:15.520 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:15.520 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 38 M 2024-09-17T08:25:15.520 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 212 M 2024-09-17T08:25:15.520 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2024-09-17T08:25:15.567 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-09-17T08:25:15.567 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:15.567 INFO:teuthology.orchestra.run.smithi118.stdout:Installed: 2024-09-17T08:25:15.567 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-grafana-dashboards-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-17T08:25:15.567 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-dashboard-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-17T08:25:15.567 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-prometheus-alerts-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-17T08:25:15.567 INFO:teuthology.orchestra.run.smithi118.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-17T08:25:15.568 INFO:teuthology.orchestra.run.smithi118.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-17T08:25:15.568 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:15.568 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2024-09-17T08:25:15.807 INFO:teuthology.orchestra.run.smithi170.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 191 kB 00:00 2024-09-17T08:25:15.819 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-17T08:25:15.849 INFO:teuthology.orchestra.run.smithi170.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.2 MB/s | 801 kB 00:00 2024-09-17T08:25:15.874 INFO:teuthology.orchestra.run.smithi170.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 488 kB/s | 33 kB 00:00 2024-09-17T08:25:15.916 INFO:teuthology.orchestra.run.smithi170.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 436 kB/s | 18 kB 00:00 2024-09-17T08:25:15.975 INFO:teuthology.orchestra.run.smithi170.stdout:(5/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 24 MB/s | 3.0 MB 00:00 2024-09-17T08:25:16.000 INFO:teuthology.orchestra.run.smithi170.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 459 kB/s | 38 kB 00:00 2024-09-17T08:25:16.092 INFO:teuthology.orchestra.run.smithi170.stdout:(7/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 42 MB/s | 4.9 MB 00:00 2024-09-17T08:25:16.117 INFO:teuthology.orchestra.run.smithi170.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 2.0 MB/s | 245 kB 00:00 2024-09-17T08:25:16.209 INFO:teuthology.orchestra.run.smithi170.stdout:(9/12): ceph-mgr-diskprediction-local-18.2.4-85 14 MB/s | 7.4 MB 00:00 2024-09-17T08:25:16.234 INFO:teuthology.orchestra.run.smithi170.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.1 MB/s | 250 kB 00:00 2024-09-17T08:25:16.310 INFO:teuthology.orchestra.run.smithi170.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 5.1 MB 00:00 2024-09-17T08:25:16.394 INFO:teuthology.orchestra.run.smithi118.stdout:Last metadata expiration check: 0:00:59 ago on Tue 17 Sep 2024 08:24:17 AM UTC. 2024-09-17T08:25:16.526 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2024-09-17T08:25:16.527 INFO:teuthology.orchestra.run.smithi118.stdout:====================================================================================== 2024-09-17T08:25:16.527 INFO:teuthology.orchestra.run.smithi118.stdout: Package Arch Version Repository Size 2024-09-17T08:25:16.527 INFO:teuthology.orchestra.run.smithi118.stdout:====================================================================================== 2024-09-17T08:25:16.527 INFO:teuthology.orchestra.run.smithi118.stdout:Installing: 2024-09-17T08:25:16.527 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 7.4 M 2024-09-17T08:25:16.527 INFO:teuthology.orchestra.run.smithi118.stdout:Installing dependencies: 2024-09-17T08:25:16.527 INFO:teuthology.orchestra.run.smithi118.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-17T08:25:16.527 INFO:teuthology.orchestra.run.smithi118.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-17T08:25:16.527 INFO:teuthology.orchestra.run.smithi118.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-17T08:25:16.528 INFO:teuthology.orchestra.run.smithi118.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-17T08:25:16.528 INFO:teuthology.orchestra.run.smithi118.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-17T08:25:16.528 INFO:teuthology.orchestra.run.smithi118.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-17T08:25:16.528 INFO:teuthology.orchestra.run.smithi118.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-17T08:25:16.528 INFO:teuthology.orchestra.run.smithi118.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-17T08:25:16.528 INFO:teuthology.orchestra.run.smithi118.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-17T08:25:16.528 INFO:teuthology.orchestra.run.smithi118.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-17T08:25:16.528 INFO:teuthology.orchestra.run.smithi118.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-17T08:25:16.528 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:16.528 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction Summary 2024-09-17T08:25:16.528 INFO:teuthology.orchestra.run.smithi118.stdout:====================================================================================== 2024-09-17T08:25:16.528 INFO:teuthology.orchestra.run.smithi118.stdout:Install 12 Packages 2024-09-17T08:25:16.529 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:16.529 INFO:teuthology.orchestra.run.smithi118.stdout:Total download size: 38 M 2024-09-17T08:25:16.530 INFO:teuthology.orchestra.run.smithi118.stdout:Installed size: 212 M 2024-09-17T08:25:16.530 INFO:teuthology.orchestra.run.smithi118.stdout:Downloading Packages: 2024-09-17T08:25:16.668 INFO:teuthology.orchestra.run.smithi170.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-09-17T08:25:16.669 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:25:16.669 INFO:teuthology.orchestra.run.smithi170.stdout:Total 33 MB/s | 38 MB 00:01 2024-09-17T08:25:17.022 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2024-09-17T08:25:17.048 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2024-09-17T08:25:17.049 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2024-09-17T08:25:17.056 INFO:teuthology.orchestra.run.smithi118.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 191 kB 00:00 2024-09-17T08:25:17.114 INFO:teuthology.orchestra.run.smithi118.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 3.9 MB/s | 801 kB 00:00 2024-09-17T08:25:17.165 INFO:teuthology.orchestra.run.smithi118.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 301 kB/s | 33 kB 00:00 2024-09-17T08:25:17.198 INFO:teuthology.orchestra.run.smithi118.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 544 kB/s | 18 kB 00:00 2024-09-17T08:25:17.232 INFO:teuthology.orchestra.run.smithi118.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 1.1 MB/s | 38 kB 00:00 2024-09-17T08:25:17.374 INFO:teuthology.orchestra.run.smithi118.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 12 MB/s | 3.0 MB 00:00 2024-09-17T08:25:17.458 INFO:teuthology.orchestra.run.smithi118.stdout:(7/12): ceph-mgr-diskprediction-local-18.2.4-85 14 MB/s | 7.4 MB 00:00 2024-09-17T08:25:17.483 INFO:teuthology.orchestra.run.smithi118.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-17T08:25:17.557 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2024-09-17T08:25:17.557 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2024-09-17T08:25:17.567 INFO:teuthology.orchestra.run.smithi118.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 15 MB/s | 4.9 MB 00:00 2024-09-17T08:25:17.600 INFO:teuthology.orchestra.run.smithi118.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.1 MB/s | 250 kB 00:00 2024-09-17T08:25:17.901 INFO:teuthology.orchestra.run.smithi118.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 49 MB/s | 16 MB 00:00 2024-09-17T08:25:18.259 INFO:teuthology.orchestra.run.smithi118.stdout:(12/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.4 MB/s | 5.1 MB 00:00 2024-09-17T08:25:18.260 INFO:teuthology.orchestra.run.smithi118.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:25:18.260 INFO:teuthology.orchestra.run.smithi118.stdout:Total 22 MB/s | 38 MB 00:01 2024-09-17T08:25:18.325 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2024-09-17T08:25:18.406 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-17T08:25:18.453 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-17T08:25:18.548 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-17T08:25:18.593 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-17T08:25:18.609 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction check 2024-09-17T08:25:18.634 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction check succeeded. 2024-09-17T08:25:18.634 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction test 2024-09-17T08:25:18.777 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-17T08:25:18.816 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-17T08:25:18.893 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-17T08:25:19.139 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction test succeeded. 2024-09-17T08:25:19.139 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction 2024-09-17T08:25:19.546 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-17T08:25:19.655 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-17T08:25:19.901 INFO:teuthology.orchestra.run.smithi118.stdout: Preparing : 1/1 2024-09-17T08:25:20.006 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-17T08:25:20.064 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-17T08:25:20.167 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-17T08:25:20.220 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-17T08:25:20.402 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-17T08:25:20.433 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-17T08:25:20.510 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-17T08:25:20.928 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T08:25:21.167 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-17T08:25:21.279 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-17T08:25:21.979 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-17T08:25:22.008 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.4-855.g19e2a2 12/12 2024-09-17T08:25:22.550 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T08:25:23.587 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-17T08:25:23.609 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.4-855.g19e2a2 12/12 2024-09-17T08:25:24.725 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.4-855.g19e2a2 12/12 2024-09-17T08:25:24.725 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.4-855.g19e2a2 1/12 2024-09-17T08:25:24.725 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-17T08:25:24.725 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-17T08:25:24.725 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-17T08:25:24.725 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-17T08:25:24.726 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-17T08:25:24.726 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-17T08:25:24.726 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-17T08:25:24.726 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-17T08:25:24.726 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T08:25:24.726 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-17T08:25:25.141 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-17T08:25:25.141 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:25.141 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2024-09-17T08:25:25.141 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-diskprediction-local-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-17T08:25:25.141 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-17T08:25:25.141 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-17T08:25:25.141 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-17T08:25:25.141 INFO:teuthology.orchestra.run.smithi170.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-17T08:25:25.141 INFO:teuthology.orchestra.run.smithi170.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-17T08:25:25.141 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-17T08:25:25.141 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-17T08:25:25.142 INFO:teuthology.orchestra.run.smithi170.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-17T08:25:25.142 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-17T08:25:25.142 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-17T08:25:25.142 INFO:teuthology.orchestra.run.smithi170.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-17T08:25:25.142 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:25.142 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-09-17T08:25:25.523 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph-mgr-rook 2024-09-17T08:25:26.109 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:01:11 ago on Tue 17 Sep 2024 08:24:15 AM UTC. 2024-09-17T08:25:26.241 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-09-17T08:25:26.243 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:25:26.243 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2024-09-17T08:25:26.243 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:25:26.243 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2024-09-17T08:25:26.243 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-rook noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 49 k 2024-09-17T08:25:26.243 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2024-09-17T08:25:26.243 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-17T08:25:26.243 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-17T08:25:26.243 INFO:teuthology.orchestra.run.smithi170.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-17T08:25:26.244 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-17T08:25:26.244 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-17T08:25:26.244 INFO:teuthology.orchestra.run.smithi170.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-17T08:25:26.244 INFO:teuthology.orchestra.run.smithi170.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-17T08:25:26.244 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-17T08:25:26.244 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-17T08:25:26.244 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-17T08:25:26.244 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-17T08:25:26.244 INFO:teuthology.orchestra.run.smithi170.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-17T08:25:26.244 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:26.244 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2024-09-17T08:25:26.244 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:25:26.244 INFO:teuthology.orchestra.run.smithi170.stdout:Install 13 Packages 2024-09-17T08:25:26.245 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:26.245 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 2.2 M 2024-09-17T08:25:26.245 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 26 M 2024-09-17T08:25:26.245 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2024-09-17T08:25:26.513 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.4-855.g19e2a2 12/12 2024-09-17T08:25:26.513 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.4-855.g19e2a2 1/12 2024-09-17T08:25:26.513 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-17T08:25:26.513 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-17T08:25:26.513 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-17T08:25:26.514 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-17T08:25:26.514 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-17T08:25:26.514 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-17T08:25:26.514 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-17T08:25:26.514 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-17T08:25:26.514 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T08:25:26.514 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-17T08:25:26.768 INFO:teuthology.orchestra.run.smithi170.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 169 kB/s | 19 kB 00:00 2024-09-17T08:25:26.794 INFO:teuthology.orchestra.run.smithi170.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 185 kB/s | 26 kB 00:00 2024-09-17T08:25:26.818 INFO:teuthology.orchestra.run.smithi170.stdout:(3/13): ceph-mgr-rook-18.2.4-855.g19e2a2be.el9. 300 kB/s | 49 kB 00:00 2024-09-17T08:25:26.913 INFO:teuthology.orchestra.run.smithi170.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-17T08:25:26.936 INFO:teuthology.orchestra.run.smithi170.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.3 MB/s | 222 kB 00:00 2024-09-17T08:25:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-17T08:25:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:26.939 INFO:teuthology.orchestra.run.smithi118.stdout:Installed: 2024-09-17T08:25:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-diskprediction-local-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-17T08:25:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-17T08:25:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-17T08:25:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-17T08:25:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-17T08:25:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-17T08:25:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-17T08:25:26.940 INFO:teuthology.orchestra.run.smithi118.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-17T08:25:26.940 INFO:teuthology.orchestra.run.smithi118.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-17T08:25:26.940 INFO:teuthology.orchestra.run.smithi118.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-17T08:25:26.940 INFO:teuthology.orchestra.run.smithi118.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-17T08:25:26.940 INFO:teuthology.orchestra.run.smithi118.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-17T08:25:26.940 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:26.940 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2024-09-17T08:25:26.969 INFO:teuthology.orchestra.run.smithi170.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 914 kB/s | 54 kB 00:00 2024-09-17T08:25:27.011 INFO:teuthology.orchestra.run.smithi170.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.4 MB/s | 279 kB 00:00 2024-09-17T08:25:27.037 INFO:teuthology.orchestra.run.smithi170.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 319 kB/s | 32 kB 00:00 2024-09-17T08:25:27.070 INFO:teuthology.orchestra.run.smithi170.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 140 kB/s | 14 kB 00:00 2024-09-17T08:25:27.095 INFO:teuthology.orchestra.run.smithi170.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.5 MB/s | 59 kB 00:00 2024-09-17T08:25:27.137 INFO:teuthology.orchestra.run.smithi170.stdout:(11/13): python3-kubernetes-26.1.0-2.el9.noarch 11 MB/s | 1.0 MB 00:00 2024-09-17T08:25:27.163 INFO:teuthology.orchestra.run.smithi170.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.3 MB/s | 90 kB 00:00 2024-09-17T08:25:27.188 INFO:teuthology.orchestra.run.smithi170.stdout:(13/13): python3-google-auth-2.34.0-1.el9.noarc 1.2 MB/s | 220 kB 00:00 2024-09-17T08:25:27.191 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:25:27.191 INFO:teuthology.orchestra.run.smithi170.stdout:Total 2.3 MB/s | 2.2 MB 00:00 2024-09-17T08:25:27.255 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2024-09-17T08:25:27.270 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2024-09-17T08:25:27.270 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2024-09-17T08:25:27.298 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum -y install ceph-mgr-rook 2024-09-17T08:25:27.357 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2024-09-17T08:25:27.357 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2024-09-17T08:25:27.865 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2024-09-17T08:25:27.887 INFO:teuthology.orchestra.run.smithi118.stdout:Last metadata expiration check: 0:01:10 ago on Tue 17 Sep 2024 08:24:17 AM UTC. 2024-09-17T08:25:28.019 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2024-09-17T08:25:28.020 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:25:28.020 INFO:teuthology.orchestra.run.smithi118.stdout: Package Arch Version Repository Size 2024-09-17T08:25:28.020 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:25:28.020 INFO:teuthology.orchestra.run.smithi118.stdout:Installing: 2024-09-17T08:25:28.020 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-rook noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 49 k 2024-09-17T08:25:28.020 INFO:teuthology.orchestra.run.smithi118.stdout:Installing dependencies: 2024-09-17T08:25:28.020 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-17T08:25:28.020 INFO:teuthology.orchestra.run.smithi118.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-17T08:25:28.020 INFO:teuthology.orchestra.run.smithi118.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-17T08:25:28.020 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-17T08:25:28.021 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-17T08:25:28.021 INFO:teuthology.orchestra.run.smithi118.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-17T08:25:28.021 INFO:teuthology.orchestra.run.smithi118.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-17T08:25:28.021 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-17T08:25:28.021 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-17T08:25:28.021 INFO:teuthology.orchestra.run.smithi118.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-17T08:25:28.021 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-17T08:25:28.021 INFO:teuthology.orchestra.run.smithi118.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-17T08:25:28.021 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:28.021 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction Summary 2024-09-17T08:25:28.021 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:25:28.021 INFO:teuthology.orchestra.run.smithi118.stdout:Install 13 Packages 2024-09-17T08:25:28.021 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:28.022 INFO:teuthology.orchestra.run.smithi118.stdout:Total download size: 2.2 M 2024-09-17T08:25:28.022 INFO:teuthology.orchestra.run.smithi118.stdout:Installed size: 26 M 2024-09-17T08:25:28.022 INFO:teuthology.orchestra.run.smithi118.stdout:Downloading Packages: 2024-09-17T08:25:28.027 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-17T08:25:28.081 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-17T08:25:28.136 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-17T08:25:28.168 INFO:teuthology.orchestra.run.smithi118.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 357 kB/s | 26 kB 00:00 2024-09-17T08:25:28.173 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-17T08:25:28.193 INFO:teuthology.orchestra.run.smithi118.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 197 kB/s | 19 kB 00:00 2024-09-17T08:25:28.215 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-17T08:25:28.269 INFO:teuthology.orchestra.run.smithi118.stdout:(3/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-09-17T08:25:28.294 INFO:teuthology.orchestra.run.smithi118.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.2 MB/s | 222 kB 00:00 2024-09-17T08:25:28.316 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-17T08:25:28.319 INFO:teuthology.orchestra.run.smithi118.stdout:(5/13): ceph-mgr-rook-18.2.4-855.g19e2a2be.el9. 221 kB/s | 49 kB 00:00 2024-09-17T08:25:28.345 INFO:teuthology.orchestra.run.smithi118.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-09-17T08:25:28.370 INFO:teuthology.orchestra.run.smithi118.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.7 MB/s | 279 kB 00:00 2024-09-17T08:25:28.414 INFO:teuthology.orchestra.run.smithi118.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 348 kB/s | 32 kB 00:00 2024-09-17T08:25:28.426 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-17T08:25:28.437 INFO:teuthology.orchestra.run.smithi118.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 153 kB/s | 14 kB 00:00 2024-09-17T08:25:28.462 INFO:teuthology.orchestra.run.smithi118.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-17T08:25:28.483 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-17T08:25:28.488 INFO:teuthology.orchestra.run.smithi118.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 2.1 MB/s | 220 kB 00:00 2024-09-17T08:25:28.513 INFO:teuthology.orchestra.run.smithi118.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.7 MB/s | 90 kB 00:00 2024-09-17T08:25:28.546 INFO:teuthology.orchestra.run.smithi118.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.7 MB/s | 1.0 MB 00:00 2024-09-17T08:25:28.549 INFO:teuthology.orchestra.run.smithi118.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:25:28.549 INFO:teuthology.orchestra.run.smithi118.stdout:Total 4.2 MB/s | 2.2 MB 00:00 2024-09-17T08:25:28.611 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction check 2024-09-17T08:25:28.627 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction check succeeded. 2024-09-17T08:25:28.627 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction test 2024-09-17T08:25:28.715 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction test succeeded. 2024-09-17T08:25:28.715 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction 2024-09-17T08:25:29.201 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-17T08:25:29.228 INFO:teuthology.orchestra.run.smithi118.stdout: Preparing : 1/1 2024-09-17T08:25:29.255 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-17T08:25:29.289 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-17T08:25:29.338 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-17T08:25:29.368 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-rook-2:18.2.4-855.g19e2a2be.el9.noarch 13/13 2024-09-17T08:25:29.412 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-17T08:25:29.473 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-17T08:25:29.537 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-17T08:25:29.574 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-17T08:25:29.616 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-17T08:25:29.726 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-17T08:25:29.844 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-17T08:25:29.892 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-17T08:25:30.161 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.4-855.g19e2a2be.el9.noarch 13/13 2024-09-17T08:25:30.162 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-rook-2:18.2.4-855.g19e2a2be.el9.noarch 1/13 2024-09-17T08:25:30.162 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-17T08:25:30.162 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-17T08:25:30.162 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-17T08:25:30.162 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-17T08:25:30.162 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-17T08:25:30.162 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-17T08:25:30.162 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-17T08:25:30.162 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-17T08:25:30.162 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-17T08:25:30.162 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-17T08:25:30.163 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-17T08:25:30.500 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-17T08:25:30.500 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:30.500 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2024-09-17T08:25:30.500 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-rook-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-17T08:25:30.500 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-17T08:25:30.500 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-17T08:25:30.500 INFO:teuthology.orchestra.run.smithi170.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-17T08:25:30.500 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-17T08:25:30.500 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-17T08:25:30.501 INFO:teuthology.orchestra.run.smithi170.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-17T08:25:30.501 INFO:teuthology.orchestra.run.smithi170.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-17T08:25:30.501 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-17T08:25:30.501 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-17T08:25:30.501 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-17T08:25:30.501 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-17T08:25:30.501 INFO:teuthology.orchestra.run.smithi170.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-17T08:25:30.501 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:30.501 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-09-17T08:25:30.601 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-17T08:25:30.657 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-17T08:25:30.691 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-17T08:25:30.739 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-17T08:25:30.754 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph-mgr-cephadm 2024-09-17T08:25:30.777 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mgr-rook-2:18.2.4-855.g19e2a2be.el9.noarch 13/13 2024-09-17T08:25:31.344 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:01:16 ago on Tue 17 Sep 2024 08:24:15 AM UTC. 2024-09-17T08:25:31.477 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-09-17T08:25:31.478 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:25:31.478 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2024-09-17T08:25:31.478 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:25:31.478 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2024-09-17T08:25:31.478 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-cephadm noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 139 k 2024-09-17T08:25:31.478 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2024-09-17T08:25:31.479 INFO:teuthology.orchestra.run.smithi170.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-17T08:25:31.479 INFO:teuthology.orchestra.run.smithi170.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-17T08:25:31.479 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-17T08:25:31.479 INFO:teuthology.orchestra.run.smithi170.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-17T08:25:31.479 INFO:teuthology.orchestra.run.smithi170.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-17T08:25:31.479 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:31.479 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2024-09-17T08:25:31.479 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:25:31.479 INFO:teuthology.orchestra.run.smithi170.stdout:Install 6 Packages 2024-09-17T08:25:31.479 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:31.480 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 7.0 M 2024-09-17T08:25:31.480 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 33 M 2024-09-17T08:25:31.480 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2024-09-17T08:25:31.561 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.4-855.g19e2a2be.el9.noarch 13/13 2024-09-17T08:25:31.561 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mgr-rook-2:18.2.4-855.g19e2a2be.el9.noarch 1/13 2024-09-17T08:25:31.561 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-17T08:25:31.561 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-17T08:25:31.561 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-17T08:25:31.561 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-17T08:25:31.561 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-17T08:25:31.562 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-17T08:25:31.562 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-17T08:25:31.562 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-17T08:25:31.562 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-17T08:25:31.562 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-17T08:25:31.562 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-17T08:25:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-17T08:25:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:31.896 INFO:teuthology.orchestra.run.smithi118.stdout:Installed: 2024-09-17T08:25:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-rook-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-17T08:25:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-17T08:25:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-17T08:25:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-17T08:25:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-17T08:25:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-17T08:25:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-17T08:25:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-17T08:25:31.897 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-17T08:25:31.897 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-17T08:25:31.897 INFO:teuthology.orchestra.run.smithi118.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-17T08:25:31.897 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-17T08:25:31.897 INFO:teuthology.orchestra.run.smithi118.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-17T08:25:31.897 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:31.897 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2024-09-17T08:25:32.091 INFO:teuthology.orchestra.run.smithi170.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.7 MB/s | 248 kB 00:00 2024-09-17T08:25:32.149 INFO:teuthology.orchestra.run.smithi170.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 9.2 MB/s | 548 kB 00:00 2024-09-17T08:25:32.174 INFO:teuthology.orchestra.run.smithi170.stdout:(3/6): ceph-mgr-cephadm-18.2.4-855.g19e2a2be.el 616 kB/s | 139 kB 00:00 2024-09-17T08:25:32.176 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum -y install ceph-mgr-cephadm 2024-09-17T08:25:32.200 INFO:teuthology.orchestra.run.smithi170.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-09-17T08:25:32.242 INFO:teuthology.orchestra.run.smithi170.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.1 MB/s | 74 kB 00:00 2024-09-17T08:25:32.317 INFO:teuthology.orchestra.run.smithi170.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-17T08:25:32.320 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:25:32.320 INFO:teuthology.orchestra.run.smithi170.stdout:Total 8.3 MB/s | 7.0 MB 00:00 2024-09-17T08:25:32.390 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2024-09-17T08:25:32.401 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2024-09-17T08:25:32.401 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2024-09-17T08:25:32.500 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2024-09-17T08:25:32.501 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2024-09-17T08:25:32.779 INFO:teuthology.orchestra.run.smithi118.stdout:Last metadata expiration check: 0:01:15 ago on Tue 17 Sep 2024 08:24:17 AM UTC. 2024-09-17T08:25:32.802 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2024-09-17T08:25:32.911 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2024-09-17T08:25:32.912 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:25:32.912 INFO:teuthology.orchestra.run.smithi118.stdout: Package Arch Version Repository Size 2024-09-17T08:25:32.912 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:25:32.912 INFO:teuthology.orchestra.run.smithi118.stdout:Installing: 2024-09-17T08:25:32.912 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-cephadm noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 139 k 2024-09-17T08:25:32.912 INFO:teuthology.orchestra.run.smithi118.stdout:Installing dependencies: 2024-09-17T08:25:32.912 INFO:teuthology.orchestra.run.smithi118.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-17T08:25:32.913 INFO:teuthology.orchestra.run.smithi118.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-17T08:25:32.913 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-17T08:25:32.913 INFO:teuthology.orchestra.run.smithi118.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-17T08:25:32.913 INFO:teuthology.orchestra.run.smithi118.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-17T08:25:32.913 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:32.913 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction Summary 2024-09-17T08:25:32.913 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:25:32.913 INFO:teuthology.orchestra.run.smithi118.stdout:Install 6 Packages 2024-09-17T08:25:32.913 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:32.914 INFO:teuthology.orchestra.run.smithi118.stdout:Total download size: 7.0 M 2024-09-17T08:25:32.914 INFO:teuthology.orchestra.run.smithi118.stdout:Installed size: 33 M 2024-09-17T08:25:32.914 INFO:teuthology.orchestra.run.smithi118.stdout:Downloading Packages: 2024-09-17T08:25:32.977 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-17T08:25:33.036 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-17T08:25:33.184 INFO:teuthology.orchestra.run.smithi118.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.3 MB/s | 248 kB 00:00 2024-09-17T08:25:33.218 INFO:teuthology.orchestra.run.smithi118.stdout:(2/6): ceph-mgr-cephadm-18.2.4-855.g19e2a2be.el 615 kB/s | 139 kB 00:00 2024-09-17T08:25:33.269 INFO:teuthology.orchestra.run.smithi118.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 6.4 MB/s | 548 kB 00:00 2024-09-17T08:25:33.294 INFO:teuthology.orchestra.run.smithi118.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 766 kB/s | 58 kB 00:00 2024-09-17T08:25:33.319 INFO:teuthology.orchestra.run.smithi118.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.5 MB/s | 74 kB 00:00 2024-09-17T08:25:33.419 INFO:teuthology.orchestra.run.smithi118.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-09-17T08:25:33.422 INFO:teuthology.orchestra.run.smithi118.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:25:33.422 INFO:teuthology.orchestra.run.smithi118.stdout:Total 14 MB/s | 7.0 MB 00:00 2024-09-17T08:25:33.493 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction check 2024-09-17T08:25:33.504 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction check succeeded. 2024-09-17T08:25:33.504 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction test 2024-09-17T08:25:33.569 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-17T08:25:33.604 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction test succeeded. 2024-09-17T08:25:33.605 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction 2024-09-17T08:25:33.656 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-17T08:25:33.717 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-17T08:25:33.748 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 6/6 2024-09-17T08:25:33.909 INFO:teuthology.orchestra.run.smithi118.stdout: Preparing : 1/1 2024-09-17T08:25:34.120 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-17T08:25:34.171 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-17T08:25:34.530 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 6/6 2024-09-17T08:25:34.530 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/6 2024-09-17T08:25:34.531 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-17T08:25:34.531 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-17T08:25:34.531 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-17T08:25:34.531 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-17T08:25:34.690 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-17T08:25:34.774 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-17T08:25:34.799 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-17T08:25:34.799 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:34.799 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2024-09-17T08:25:34.799 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-17T08:25:34.799 INFO:teuthology.orchestra.run.smithi170.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-17T08:25:34.800 INFO:teuthology.orchestra.run.smithi170.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-17T08:25:34.800 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-17T08:25:34.800 INFO:teuthology.orchestra.run.smithi170.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-17T08:25:34.800 INFO:teuthology.orchestra.run.smithi170.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-17T08:25:34.800 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:34.800 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-09-17T08:25:34.845 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-17T08:25:34.875 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mgr-cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 6/6 2024-09-17T08:25:35.040 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph-fuse 2024-09-17T08:25:35.622 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:01:20 ago on Tue 17 Sep 2024 08:24:15 AM UTC. 2024-09-17T08:25:35.688 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 6/6 2024-09-17T08:25:35.688 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mgr-cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/6 2024-09-17T08:25:35.688 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-17T08:25:35.688 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-17T08:25:35.688 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-17T08:25:35.688 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-17T08:25:35.754 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-09-17T08:25:35.754 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:25:35.754 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2024-09-17T08:25:35.755 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:25:35.755 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2024-09-17T08:25:35.755 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-fuse x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 852 k 2024-09-17T08:25:35.755 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2024-09-17T08:25:35.755 INFO:teuthology.orchestra.run.smithi170.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-17T08:25:35.755 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:35.755 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2024-09-17T08:25:35.755 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:25:35.755 INFO:teuthology.orchestra.run.smithi170.stdout:Install 2 Packages 2024-09-17T08:25:35.755 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:35.756 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 932 k 2024-09-17T08:25:35.756 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 2.7 M 2024-09-17T08:25:35.756 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2024-09-17T08:25:35.949 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-17T08:25:35.949 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:35.949 INFO:teuthology.orchestra.run.smithi118.stdout:Installed: 2024-09-17T08:25:35.949 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-17T08:25:35.949 INFO:teuthology.orchestra.run.smithi118.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-17T08:25:35.949 INFO:teuthology.orchestra.run.smithi118.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-17T08:25:35.949 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-17T08:25:35.949 INFO:teuthology.orchestra.run.smithi118.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-17T08:25:35.949 INFO:teuthology.orchestra.run.smithi118.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-17T08:25:35.949 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:35.950 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2024-09-17T08:25:36.028 INFO:teuthology.orchestra.run.smithi170.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 563 kB/s | 80 kB 00:00 2024-09-17T08:25:36.189 INFO:teuthology.orchestra.run.smithi170.stdout:(2/2): ceph-fuse-18.2.4-855.g19e2a2be.el9.x86_6 2.8 MB/s | 852 kB 00:00 2024-09-17T08:25:36.189 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:25:36.189 INFO:teuthology.orchestra.run.smithi170.stdout:Total 2.1 MB/s | 932 kB 00:00 2024-09-17T08:25:36.201 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2024-09-17T08:25:36.217 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum -y install ceph-fuse 2024-09-17T08:25:36.223 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2024-09-17T08:25:36.223 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2024-09-17T08:25:36.303 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2024-09-17T08:25:36.304 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2024-09-17T08:25:36.495 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2024-09-17T08:25:36.619 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-17T08:25:36.685 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 2/2 2024-09-17T08:25:36.798 INFO:teuthology.orchestra.run.smithi118.stdout:Last metadata expiration check: 0:01:19 ago on Tue 17 Sep 2024 08:24:17 AM UTC. 2024-09-17T08:25:36.932 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2024-09-17T08:25:36.932 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:25:36.932 INFO:teuthology.orchestra.run.smithi118.stdout: Package Arch Version Repository Size 2024-09-17T08:25:36.933 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:25:36.933 INFO:teuthology.orchestra.run.smithi118.stdout:Installing: 2024-09-17T08:25:36.933 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-fuse x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 852 k 2024-09-17T08:25:36.933 INFO:teuthology.orchestra.run.smithi118.stdout:Installing dependencies: 2024-09-17T08:25:36.933 INFO:teuthology.orchestra.run.smithi118.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-17T08:25:36.933 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:36.933 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction Summary 2024-09-17T08:25:36.933 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:25:36.933 INFO:teuthology.orchestra.run.smithi118.stdout:Install 2 Packages 2024-09-17T08:25:36.933 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:36.934 INFO:teuthology.orchestra.run.smithi118.stdout:Total download size: 932 k 2024-09-17T08:25:36.934 INFO:teuthology.orchestra.run.smithi118.stdout:Installed size: 2.7 M 2024-09-17T08:25:36.934 INFO:teuthology.orchestra.run.smithi118.stdout:Downloading Packages: 2024-09-17T08:25:37.239 INFO:teuthology.orchestra.run.smithi118.stdout:(1/2): ceph-fuse-18.2.4-855.g19e2a2be.el9.x86_6 2.9 MB/s | 852 kB 00:00 2024-09-17T08:25:37.399 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 2/2 2024-09-17T08:25:37.400 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 1/2 2024-09-17T08:25:37.597 INFO:teuthology.orchestra.run.smithi118.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 124 kB/s | 80 kB 00:00 2024-09-17T08:25:37.598 INFO:teuthology.orchestra.run.smithi118.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:25:37.598 INFO:teuthology.orchestra.run.smithi118.stdout:Total 1.4 MB/s | 932 kB 00:00 2024-09-17T08:25:37.610 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction check 2024-09-17T08:25:37.633 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction check succeeded. 2024-09-17T08:25:37.633 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction test 2024-09-17T08:25:37.686 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-17T08:25:37.686 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:37.686 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2024-09-17T08:25:37.687 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-17T08:25:37.687 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:37.687 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-09-17T08:25:37.714 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction test succeeded. 2024-09-17T08:25:37.714 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction 2024-09-17T08:25:37.901 INFO:teuthology.orchestra.run.smithi118.stdout: Preparing : 1/1 2024-09-17T08:25:37.949 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph-volume 2024-09-17T08:25:38.037 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-17T08:25:38.104 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 2/2 2024-09-17T08:25:38.539 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:01:23 ago on Tue 17 Sep 2024 08:24:15 AM UTC. 2024-09-17T08:25:38.679 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-09-17T08:25:38.679 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:25:38.679 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2024-09-17T08:25:38.679 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:25:38.680 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2024-09-17T08:25:38.680 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-volume noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 264 k 2024-09-17T08:25:38.680 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2024-09-17T08:25:38.680 INFO:teuthology.orchestra.run.smithi170.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-17T08:25:38.680 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:38.680 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2024-09-17T08:25:38.680 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:25:38.680 INFO:teuthology.orchestra.run.smithi170.stdout:Install 2 Packages 2024-09-17T08:25:38.680 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:38.680 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 342 k 2024-09-17T08:25:38.680 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 1.5 M 2024-09-17T08:25:38.680 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2024-09-17T08:25:38.766 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 2/2 2024-09-17T08:25:38.767 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 1/2 2024-09-17T08:25:38.959 INFO:teuthology.orchestra.run.smithi170.stdout:(1/2): ceph-volume-18.2.4-855.g19e2a2be.el9.noa 1.1 MB/s | 264 kB 00:00 2024-09-17T08:25:39.034 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-17T08:25:39.034 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:39.034 INFO:teuthology.orchestra.run.smithi118.stdout:Installed: 2024-09-17T08:25:39.034 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-17T08:25:39.034 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:39.034 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2024-09-17T08:25:39.117 INFO:teuthology.orchestra.run.smithi170.stdout:(2/2): python3-packaging-20.9-5.el9.noarch.rpm 199 kB/s | 77 kB 00:00 2024-09-17T08:25:39.118 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:25:39.118 INFO:teuthology.orchestra.run.smithi170.stdout:Total 779 kB/s | 342 kB 00:00 2024-09-17T08:25:39.130 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2024-09-17T08:25:39.139 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2024-09-17T08:25:39.139 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2024-09-17T08:25:39.177 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2024-09-17T08:25:39.177 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2024-09-17T08:25:39.323 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2024-09-17T08:25:39.333 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum -y install ceph-volume 2024-09-17T08:25:39.518 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-09-17T08:25:39.562 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-volume-2:18.2.4-855.g19e2a2be.el9.noarch 2/2 2024-09-17T08:25:39.578 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-volume-2:18.2.4-855.g19e2a2be.el9.noarch 2/2 2024-09-17T08:25:39.578 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T08:25:39.578 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-17T08:25:39.578 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:39.927 INFO:teuthology.orchestra.run.smithi118.stdout:Last metadata expiration check: 0:01:22 ago on Tue 17 Sep 2024 08:24:17 AM UTC. 2024-09-17T08:25:40.068 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2024-09-17T08:25:40.068 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:25:40.069 INFO:teuthology.orchestra.run.smithi118.stdout: Package Arch Version Repository Size 2024-09-17T08:25:40.069 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:25:40.069 INFO:teuthology.orchestra.run.smithi118.stdout:Installing: 2024-09-17T08:25:40.069 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-volume noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 264 k 2024-09-17T08:25:40.069 INFO:teuthology.orchestra.run.smithi118.stdout:Installing dependencies: 2024-09-17T08:25:40.069 INFO:teuthology.orchestra.run.smithi118.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-17T08:25:40.069 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:40.069 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction Summary 2024-09-17T08:25:40.069 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:25:40.069 INFO:teuthology.orchestra.run.smithi118.stdout:Install 2 Packages 2024-09-17T08:25:40.069 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:40.069 INFO:teuthology.orchestra.run.smithi118.stdout:Total download size: 342 k 2024-09-17T08:25:40.069 INFO:teuthology.orchestra.run.smithi118.stdout:Installed size: 1.5 M 2024-09-17T08:25:40.070 INFO:teuthology.orchestra.run.smithi118.stdout:Downloading Packages: 2024-09-17T08:25:40.186 INFO:teuthology.orchestra.run.smithi118.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 826 kB/s | 77 kB 00:00 2024-09-17T08:25:40.337 INFO:teuthology.orchestra.run.smithi118.stdout:(2/2): ceph-volume-18.2.4-855.g19e2a2be.el9.noa 1.1 MB/s | 264 kB 00:00 2024-09-17T08:25:40.337 INFO:teuthology.orchestra.run.smithi118.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:25:40.338 INFO:teuthology.orchestra.run.smithi118.stdout:Total 1.2 MB/s | 342 kB 00:00 2024-09-17T08:25:40.343 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-volume-2:18.2.4-855.g19e2a2be.el9.noarch 1/2 2024-09-17T08:25:40.349 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction check 2024-09-17T08:25:40.359 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction check succeeded. 2024-09-17T08:25:40.359 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction test 2024-09-17T08:25:40.398 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction test succeeded. 2024-09-17T08:25:40.398 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction 2024-09-17T08:25:40.545 INFO:teuthology.orchestra.run.smithi118.stdout: Preparing : 1/1 2024-09-17T08:25:40.654 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-09-17T08:25:40.654 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:40.654 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2024-09-17T08:25:40.654 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-volume-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-17T08:25:40.654 INFO:teuthology.orchestra.run.smithi170.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-17T08:25:40.654 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:40.654 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-09-17T08:25:40.719 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-09-17T08:25:40.764 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-volume-2:18.2.4-855.g19e2a2be.el9.noarch 2/2 2024-09-17T08:25:40.780 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-volume-2:18.2.4-855.g19e2a2be.el9.noarch 2/2 2024-09-17T08:25:40.780 INFO:teuthology.orchestra.run.smithi118.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T08:25:40.780 INFO:teuthology.orchestra.run.smithi118.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-17T08:25:40.780 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:41.007 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install librados-devel 2024-09-17T08:25:41.523 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-volume-2:18.2.4-855.g19e2a2be.el9.noarch 1/2 2024-09-17T08:25:41.592 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:01:26 ago on Tue 17 Sep 2024 08:24:15 AM UTC. 2024-09-17T08:25:41.724 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-09-17T08:25:41.725 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:25:41.726 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repo Size 2024-09-17T08:25:41.726 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:25:41.726 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2024-09-17T08:25:41.726 INFO:teuthology.orchestra.run.smithi170.stdout: librados-devel x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 126 k 2024-09-17T08:25:41.726 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:41.726 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2024-09-17T08:25:41.726 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:25:41.726 INFO:teuthology.orchestra.run.smithi170.stdout:Install 1 Package 2024-09-17T08:25:41.726 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:41.726 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 126 k 2024-09-17T08:25:41.726 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 456 k 2024-09-17T08:25:41.726 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2024-09-17T08:25:41.851 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-09-17T08:25:41.851 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:41.851 INFO:teuthology.orchestra.run.smithi118.stdout:Installed: 2024-09-17T08:25:41.851 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-volume-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-17T08:25:41.851 INFO:teuthology.orchestra.run.smithi118.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-17T08:25:41.851 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:41.851 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2024-09-17T08:25:41.940 INFO:teuthology.orchestra.run.smithi170.stdout:librados-devel-18.2.4-855.g19e2a2be.el9.x86_64. 591 kB/s | 126 kB 00:00 2024-09-17T08:25:41.941 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:25:41.941 INFO:teuthology.orchestra.run.smithi170.stdout:Total 587 kB/s | 126 kB 00:00 2024-09-17T08:25:41.941 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2024-09-17T08:25:41.946 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2024-09-17T08:25:41.946 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2024-09-17T08:25:42.010 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2024-09-17T08:25:42.010 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2024-09-17T08:25:42.164 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2024-09-17T08:25:42.168 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum -y install librados-devel 2024-09-17T08:25:42.265 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librados-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:25:42.739 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librados-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:25:42.766 INFO:teuthology.orchestra.run.smithi118.stdout:Last metadata expiration check: 0:01:25 ago on Tue 17 Sep 2024 08:24:17 AM UTC. 2024-09-17T08:25:42.901 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2024-09-17T08:25:42.902 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:25:42.902 INFO:teuthology.orchestra.run.smithi118.stdout: Package Arch Version Repo Size 2024-09-17T08:25:42.902 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:25:42.902 INFO:teuthology.orchestra.run.smithi118.stdout:Installing: 2024-09-17T08:25:42.902 INFO:teuthology.orchestra.run.smithi118.stdout: librados-devel x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 126 k 2024-09-17T08:25:42.902 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:42.902 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction Summary 2024-09-17T08:25:42.902 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:25:42.902 INFO:teuthology.orchestra.run.smithi118.stdout:Install 1 Package 2024-09-17T08:25:42.902 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:42.902 INFO:teuthology.orchestra.run.smithi118.stdout:Total download size: 126 k 2024-09-17T08:25:42.902 INFO:teuthology.orchestra.run.smithi118.stdout:Installed size: 456 k 2024-09-17T08:25:42.903 INFO:teuthology.orchestra.run.smithi118.stdout:Downloading Packages: 2024-09-17T08:25:43.032 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:25:43.032 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:43.032 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2024-09-17T08:25:43.032 INFO:teuthology.orchestra.run.smithi170.stdout: librados-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:25:43.032 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:43.032 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-09-17T08:25:43.094 INFO:teuthology.orchestra.run.smithi118.stdout:librados-devel-18.2.4-855.g19e2a2be.el9.x86_64. 662 kB/s | 126 kB 00:00 2024-09-17T08:25:43.094 INFO:teuthology.orchestra.run.smithi118.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:25:43.094 INFO:teuthology.orchestra.run.smithi118.stdout:Total 657 kB/s | 126 kB 00:00 2024-09-17T08:25:43.095 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction check 2024-09-17T08:25:43.100 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction check succeeded. 2024-09-17T08:25:43.100 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction test 2024-09-17T08:25:43.164 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction test succeeded. 2024-09-17T08:25:43.164 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction 2024-09-17T08:25:43.318 INFO:teuthology.orchestra.run.smithi118.stdout: Preparing : 1/1 2024-09-17T08:25:43.328 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install libcephfs2 2024-09-17T08:25:43.432 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : librados-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:25:43.861 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: librados-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:25:43.924 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:01:28 ago on Tue 17 Sep 2024 08:24:15 AM UTC. 2024-09-17T08:25:44.019 INFO:teuthology.orchestra.run.smithi170.stdout:Package libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-17T08:25:44.065 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-09-17T08:25:44.066 INFO:teuthology.orchestra.run.smithi170.stdout:Nothing to do. 2024-09-17T08:25:44.067 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-09-17T08:25:44.133 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librados-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:25:44.133 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:44.133 INFO:teuthology.orchestra.run.smithi118.stdout:Installed: 2024-09-17T08:25:44.133 INFO:teuthology.orchestra.run.smithi118.stdout: librados-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:25:44.133 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:44.133 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2024-09-17T08:25:44.139 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install libcephfs-devel 2024-09-17T08:25:44.421 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum -y install libcephfs2 2024-09-17T08:25:44.719 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:01:29 ago on Tue 17 Sep 2024 08:24:15 AM UTC. 2024-09-17T08:25:44.851 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-09-17T08:25:44.852 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:25:44.852 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repo Size 2024-09-17T08:25:44.852 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:25:44.853 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2024-09-17T08:25:44.853 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-devel x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 31 k 2024-09-17T08:25:44.853 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:44.853 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2024-09-17T08:25:44.853 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:25:44.853 INFO:teuthology.orchestra.run.smithi170.stdout:Install 1 Package 2024-09-17T08:25:44.853 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:44.853 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 31 k 2024-09-17T08:25:44.853 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 139 k 2024-09-17T08:25:44.853 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2024-09-17T08:25:44.994 INFO:teuthology.orchestra.run.smithi170.stdout:libcephfs-devel-18.2.4-855.g19e2a2be.el9.x86_64 220 kB/s | 31 kB 00:00 2024-09-17T08:25:44.994 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:25:44.995 INFO:teuthology.orchestra.run.smithi170.stdout:Total 218 kB/s | 31 kB 00:00 2024-09-17T08:25:44.995 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2024-09-17T08:25:44.998 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2024-09-17T08:25:44.998 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2024-09-17T08:25:45.013 INFO:teuthology.orchestra.run.smithi118.stdout:Last metadata expiration check: 0:01:28 ago on Tue 17 Sep 2024 08:24:17 AM UTC. 2024-09-17T08:25:45.023 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2024-09-17T08:25:45.023 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2024-09-17T08:25:45.109 INFO:teuthology.orchestra.run.smithi118.stdout:Package libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-17T08:25:45.110 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2024-09-17T08:25:45.155 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2024-09-17T08:25:45.156 INFO:teuthology.orchestra.run.smithi118.stdout:Nothing to do. 2024-09-17T08:25:45.156 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2024-09-17T08:25:45.225 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephfs-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:25:45.244 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum -y install libcephfs-devel 2024-09-17T08:25:45.615 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libcephfs-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:25:45.816 INFO:teuthology.orchestra.run.smithi118.stdout:Last metadata expiration check: 0:01:28 ago on Tue 17 Sep 2024 08:24:17 AM UTC. 2024-09-17T08:25:45.844 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephfs-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:25:45.844 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:45.844 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2024-09-17T08:25:45.844 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:25:45.844 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:45.844 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-09-17T08:25:45.947 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2024-09-17T08:25:45.948 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:25:45.948 INFO:teuthology.orchestra.run.smithi118.stdout: Package Arch Version Repo Size 2024-09-17T08:25:45.948 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:25:45.948 INFO:teuthology.orchestra.run.smithi118.stdout:Installing: 2024-09-17T08:25:45.948 INFO:teuthology.orchestra.run.smithi118.stdout: libcephfs-devel x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 31 k 2024-09-17T08:25:45.948 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:45.948 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction Summary 2024-09-17T08:25:45.948 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:25:45.948 INFO:teuthology.orchestra.run.smithi118.stdout:Install 1 Package 2024-09-17T08:25:45.949 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:45.949 INFO:teuthology.orchestra.run.smithi118.stdout:Total download size: 31 k 2024-09-17T08:25:45.949 INFO:teuthology.orchestra.run.smithi118.stdout:Installed size: 139 k 2024-09-17T08:25:45.949 INFO:teuthology.orchestra.run.smithi118.stdout:Downloading Packages: 2024-09-17T08:25:46.017 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install librados2 2024-09-17T08:25:46.090 INFO:teuthology.orchestra.run.smithi118.stdout:libcephfs-devel-18.2.4-855.g19e2a2be.el9.x86_64 219 kB/s | 31 kB 00:00 2024-09-17T08:25:46.091 INFO:teuthology.orchestra.run.smithi118.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:25:46.091 INFO:teuthology.orchestra.run.smithi118.stdout:Total 217 kB/s | 31 kB 00:00 2024-09-17T08:25:46.092 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction check 2024-09-17T08:25:46.095 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction check succeeded. 2024-09-17T08:25:46.095 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction test 2024-09-17T08:25:46.108 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction test succeeded. 2024-09-17T08:25:46.108 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction 2024-09-17T08:25:46.197 INFO:teuthology.orchestra.run.smithi118.stdout: Preparing : 1/1 2024-09-17T08:25:46.304 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libcephfs-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:25:46.594 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:01:31 ago on Tue 17 Sep 2024 08:24:15 AM UTC. 2024-09-17T08:25:46.681 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: libcephfs-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:25:46.683 INFO:teuthology.orchestra.run.smithi170.stdout:Package librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-17T08:25:46.726 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-09-17T08:25:46.727 INFO:teuthology.orchestra.run.smithi170.stdout:Nothing to do. 2024-09-17T08:25:46.727 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-09-17T08:25:46.799 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install librbd1 2024-09-17T08:25:46.930 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libcephfs-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:25:46.931 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:46.931 INFO:teuthology.orchestra.run.smithi118.stdout:Installed: 2024-09-17T08:25:46.931 INFO:teuthology.orchestra.run.smithi118.stdout: libcephfs-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:25:46.931 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:46.931 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2024-09-17T08:25:47.129 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum -y install librados2 2024-09-17T08:25:47.384 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:01:32 ago on Tue 17 Sep 2024 08:24:15 AM UTC. 2024-09-17T08:25:47.474 INFO:teuthology.orchestra.run.smithi170.stdout:Package librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-17T08:25:47.518 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-09-17T08:25:47.519 INFO:teuthology.orchestra.run.smithi170.stdout:Nothing to do. 2024-09-17T08:25:47.519 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-09-17T08:25:47.594 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install python3-rados 2024-09-17T08:25:47.702 INFO:teuthology.orchestra.run.smithi118.stdout:Last metadata expiration check: 0:01:30 ago on Tue 17 Sep 2024 08:24:17 AM UTC. 2024-09-17T08:25:47.790 INFO:teuthology.orchestra.run.smithi118.stdout:Package librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-17T08:25:47.833 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2024-09-17T08:25:47.834 INFO:teuthology.orchestra.run.smithi118.stdout:Nothing to do. 2024-09-17T08:25:47.834 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2024-09-17T08:25:47.918 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum -y install librbd1 2024-09-17T08:25:48.168 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:01:33 ago on Tue 17 Sep 2024 08:24:15 AM UTC. 2024-09-17T08:25:48.257 INFO:teuthology.orchestra.run.smithi170.stdout:Package python3-rados-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-17T08:25:48.301 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-09-17T08:25:48.301 INFO:teuthology.orchestra.run.smithi170.stdout:Nothing to do. 2024-09-17T08:25:48.302 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-09-17T08:25:48.374 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install python3-rgw 2024-09-17T08:25:48.494 INFO:teuthology.orchestra.run.smithi118.stdout:Last metadata expiration check: 0:01:31 ago on Tue 17 Sep 2024 08:24:17 AM UTC. 2024-09-17T08:25:48.582 INFO:teuthology.orchestra.run.smithi118.stdout:Package librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-17T08:25:48.625 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2024-09-17T08:25:48.626 INFO:teuthology.orchestra.run.smithi118.stdout:Nothing to do. 2024-09-17T08:25:48.626 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2024-09-17T08:25:48.710 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum -y install python3-rados 2024-09-17T08:25:48.942 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:01:33 ago on Tue 17 Sep 2024 08:24:15 AM UTC. 2024-09-17T08:25:49.031 INFO:teuthology.orchestra.run.smithi170.stdout:Package python3-rgw-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-17T08:25:49.073 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-09-17T08:25:49.074 INFO:teuthology.orchestra.run.smithi170.stdout:Nothing to do. 2024-09-17T08:25:49.074 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-09-17T08:25:49.144 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install python3-cephfs 2024-09-17T08:25:49.287 INFO:teuthology.orchestra.run.smithi118.stdout:Last metadata expiration check: 0:01:32 ago on Tue 17 Sep 2024 08:24:17 AM UTC. 2024-09-17T08:25:49.379 INFO:teuthology.orchestra.run.smithi118.stdout:Package python3-rados-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-17T08:25:49.422 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2024-09-17T08:25:49.423 INFO:teuthology.orchestra.run.smithi118.stdout:Nothing to do. 2024-09-17T08:25:49.423 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2024-09-17T08:25:49.502 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum -y install python3-rgw 2024-09-17T08:25:49.723 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:01:34 ago on Tue 17 Sep 2024 08:24:15 AM UTC. 2024-09-17T08:25:49.813 INFO:teuthology.orchestra.run.smithi170.stdout:Package python3-cephfs-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-17T08:25:49.856 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-09-17T08:25:49.857 INFO:teuthology.orchestra.run.smithi170.stdout:Nothing to do. 2024-09-17T08:25:49.857 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-09-17T08:25:49.927 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install python3-rbd 2024-09-17T08:25:50.076 INFO:teuthology.orchestra.run.smithi118.stdout:Last metadata expiration check: 0:01:33 ago on Tue 17 Sep 2024 08:24:17 AM UTC. 2024-09-17T08:25:50.167 INFO:teuthology.orchestra.run.smithi118.stdout:Package python3-rgw-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-17T08:25:50.210 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2024-09-17T08:25:50.211 INFO:teuthology.orchestra.run.smithi118.stdout:Nothing to do. 2024-09-17T08:25:50.211 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2024-09-17T08:25:50.291 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum -y install python3-cephfs 2024-09-17T08:25:50.510 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:01:35 ago on Tue 17 Sep 2024 08:24:15 AM UTC. 2024-09-17T08:25:50.602 INFO:teuthology.orchestra.run.smithi170.stdout:Package python3-rbd-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-17T08:25:50.646 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-09-17T08:25:50.647 INFO:teuthology.orchestra.run.smithi170.stdout:Nothing to do. 2024-09-17T08:25:50.647 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-09-17T08:25:50.728 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install rbd-fuse 2024-09-17T08:25:50.860 INFO:teuthology.orchestra.run.smithi118.stdout:Last metadata expiration check: 0:01:33 ago on Tue 17 Sep 2024 08:24:17 AM UTC. 2024-09-17T08:25:50.949 INFO:teuthology.orchestra.run.smithi118.stdout:Package python3-cephfs-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-17T08:25:50.992 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2024-09-17T08:25:50.993 INFO:teuthology.orchestra.run.smithi118.stdout:Nothing to do. 2024-09-17T08:25:50.993 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2024-09-17T08:25:51.074 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum -y install python3-rbd 2024-09-17T08:25:51.310 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:01:36 ago on Tue 17 Sep 2024 08:24:15 AM UTC. 2024-09-17T08:25:51.443 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-09-17T08:25:51.444 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:25:51.444 INFO:teuthology.orchestra.run.smithi170.stdout: Package Architecture Version Repository Size 2024-09-17T08:25:51.444 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:25:51.444 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2024-09-17T08:25:51.444 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-fuse x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 87 k 2024-09-17T08:25:51.444 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:51.444 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2024-09-17T08:25:51.444 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:25:51.444 INFO:teuthology.orchestra.run.smithi170.stdout:Install 1 Package 2024-09-17T08:25:51.444 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:51.444 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 87 k 2024-09-17T08:25:51.444 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 227 k 2024-09-17T08:25:51.445 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2024-09-17T08:25:51.647 INFO:teuthology.orchestra.run.smithi170.stdout:rbd-fuse-18.2.4-855.g19e2a2be.el9.x86_64.rpm 428 kB/s | 87 kB 00:00 2024-09-17T08:25:51.648 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:25:51.648 INFO:teuthology.orchestra.run.smithi170.stdout:Total 425 kB/s | 87 kB 00:00 2024-09-17T08:25:51.648 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2024-09-17T08:25:51.648 INFO:teuthology.orchestra.run.smithi118.stdout:Last metadata expiration check: 0:01:34 ago on Tue 17 Sep 2024 08:24:17 AM UTC. 2024-09-17T08:25:51.655 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2024-09-17T08:25:51.655 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2024-09-17T08:25:51.717 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2024-09-17T08:25:51.717 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2024-09-17T08:25:51.739 INFO:teuthology.orchestra.run.smithi118.stdout:Package python3-rbd-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-17T08:25:51.783 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2024-09-17T08:25:51.783 INFO:teuthology.orchestra.run.smithi118.stdout:Nothing to do. 2024-09-17T08:25:51.784 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2024-09-17T08:25:51.873 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum -y install rbd-fuse 2024-09-17T08:25:51.876 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2024-09-17T08:25:51.969 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:25:52.363 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: rbd-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:25:52.451 INFO:teuthology.orchestra.run.smithi118.stdout:Last metadata expiration check: 0:01:35 ago on Tue 17 Sep 2024 08:24:17 AM UTC. 2024-09-17T08:25:52.582 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2024-09-17T08:25:52.583 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:25:52.583 INFO:teuthology.orchestra.run.smithi118.stdout: Package Architecture Version Repository Size 2024-09-17T08:25:52.583 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:25:52.583 INFO:teuthology.orchestra.run.smithi118.stdout:Installing: 2024-09-17T08:25:52.583 INFO:teuthology.orchestra.run.smithi118.stdout: rbd-fuse x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 87 k 2024-09-17T08:25:52.583 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:52.583 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction Summary 2024-09-17T08:25:52.583 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:25:52.583 INFO:teuthology.orchestra.run.smithi118.stdout:Install 1 Package 2024-09-17T08:25:52.583 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:52.583 INFO:teuthology.orchestra.run.smithi118.stdout:Total download size: 87 k 2024-09-17T08:25:52.583 INFO:teuthology.orchestra.run.smithi118.stdout:Installed size: 227 k 2024-09-17T08:25:52.584 INFO:teuthology.orchestra.run.smithi118.stdout:Downloading Packages: 2024-09-17T08:25:52.599 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:25:52.599 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:52.599 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2024-09-17T08:25:52.600 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:25:52.600 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:52.600 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-09-17T08:25:52.785 INFO:teuthology.orchestra.run.smithi118.stdout:rbd-fuse-18.2.4-855.g19e2a2be.el9.x86_64.rpm 429 kB/s | 87 kB 00:00 2024-09-17T08:25:52.786 INFO:teuthology.orchestra.run.smithi118.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:25:52.786 INFO:teuthology.orchestra.run.smithi118.stdout:Total 427 kB/s | 87 kB 00:00 2024-09-17T08:25:52.787 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction check 2024-09-17T08:25:52.793 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction check succeeded. 2024-09-17T08:25:52.793 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction test 2024-09-17T08:25:52.856 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction test succeeded. 2024-09-17T08:25:52.856 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction 2024-09-17T08:25:52.858 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install rbd-mirror 2024-09-17T08:25:53.015 INFO:teuthology.orchestra.run.smithi118.stdout: Preparing : 1/1 2024-09-17T08:25:53.108 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : rbd-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:25:53.441 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:01:38 ago on Tue 17 Sep 2024 08:24:15 AM UTC. 2024-09-17T08:25:53.514 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: rbd-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:25:53.578 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-09-17T08:25:53.579 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:25:53.579 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repo Size 2024-09-17T08:25:53.579 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:25:53.579 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2024-09-17T08:25:53.579 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-mirror x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 3.0 M 2024-09-17T08:25:53.579 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:53.579 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2024-09-17T08:25:53.579 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:25:53.579 INFO:teuthology.orchestra.run.smithi170.stdout:Install 1 Package 2024-09-17T08:25:53.579 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:53.580 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 3.0 M 2024-09-17T08:25:53.580 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 12 M 2024-09-17T08:25:53.580 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2024-09-17T08:25:53.780 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : rbd-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:25:53.780 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:53.780 INFO:teuthology.orchestra.run.smithi118.stdout:Installed: 2024-09-17T08:25:53.780 INFO:teuthology.orchestra.run.smithi118.stdout: rbd-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:25:53.780 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:53.780 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2024-09-17T08:25:53.917 INFO:teuthology.orchestra.run.smithi170.stdout:rbd-mirror-18.2.4-855.g19e2a2be.el9.x86_64.rpm 8.9 MB/s | 3.0 MB 00:00 2024-09-17T08:25:53.918 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:25:53.918 INFO:teuthology.orchestra.run.smithi170.stdout:Total 8.8 MB/s | 3.0 MB 00:00 2024-09-17T08:25:53.918 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2024-09-17T08:25:53.927 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2024-09-17T08:25:53.927 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2024-09-17T08:25:54.007 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2024-09-17T08:25:54.008 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2024-09-17T08:25:54.036 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum -y install rbd-mirror 2024-09-17T08:25:54.442 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2024-09-17T08:25:54.510 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-mirror-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:25:54.536 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: rbd-mirror-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:25:54.536 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T08:25:54.536 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-17T08:25:54.536 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T08:25:54.536 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T08:25:54.536 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:54.619 INFO:teuthology.orchestra.run.smithi118.stdout:Last metadata expiration check: 0:01:37 ago on Tue 17 Sep 2024 08:24:17 AM UTC. 2024-09-17T08:25:54.757 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2024-09-17T08:25:54.758 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:25:54.758 INFO:teuthology.orchestra.run.smithi118.stdout: Package Arch Version Repo Size 2024-09-17T08:25:54.758 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:25:54.758 INFO:teuthology.orchestra.run.smithi118.stdout:Installing: 2024-09-17T08:25:54.758 INFO:teuthology.orchestra.run.smithi118.stdout: rbd-mirror x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 3.0 M 2024-09-17T08:25:54.758 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:54.758 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction Summary 2024-09-17T08:25:54.758 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:25:54.758 INFO:teuthology.orchestra.run.smithi118.stdout:Install 1 Package 2024-09-17T08:25:54.758 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:54.758 INFO:teuthology.orchestra.run.smithi118.stdout:Total download size: 3.0 M 2024-09-17T08:25:54.758 INFO:teuthology.orchestra.run.smithi118.stdout:Installed size: 12 M 2024-09-17T08:25:54.758 INFO:teuthology.orchestra.run.smithi118.stdout:Downloading Packages: 2024-09-17T08:25:55.089 INFO:teuthology.orchestra.run.smithi118.stdout:rbd-mirror-18.2.4-855.g19e2a2be.el9.x86_64.rpm 9.0 MB/s | 3.0 MB 00:00 2024-09-17T08:25:55.090 INFO:teuthology.orchestra.run.smithi118.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:25:55.090 INFO:teuthology.orchestra.run.smithi118.stdout:Total 9.0 MB/s | 3.0 MB 00:00 2024-09-17T08:25:55.090 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction check 2024-09-17T08:25:55.099 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction check succeeded. 2024-09-17T08:25:55.100 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction test 2024-09-17T08:25:55.179 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction test succeeded. 2024-09-17T08:25:55.179 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction 2024-09-17T08:25:55.598 INFO:teuthology.orchestra.run.smithi118.stdout: Preparing : 1/1 2024-09-17T08:25:55.657 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : rbd-mirror-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:25:55.682 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: rbd-mirror-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:25:55.682 INFO:teuthology.orchestra.run.smithi118.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T08:25:55.683 INFO:teuthology.orchestra.run.smithi118.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-17T08:25:55.683 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T08:25:55.683 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T08:25:55.683 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:55.695 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-mirror-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:25:55.695 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:55.695 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2024-09-17T08:25:55.695 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-mirror-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:25:55.695 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:55.695 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-09-17T08:25:55.953 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install rbd-nbd 2024-09-17T08:25:56.545 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:01:41 ago on Tue 17 Sep 2024 08:24:15 AM UTC. 2024-09-17T08:25:56.682 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-09-17T08:25:56.682 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:25:56.682 INFO:teuthology.orchestra.run.smithi170.stdout: Package Architecture Version Repository Size 2024-09-17T08:25:56.683 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:25:56.683 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2024-09-17T08:25:56.683 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-nbd x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 172 k 2024-09-17T08:25:56.683 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:56.683 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2024-09-17T08:25:56.683 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-09-17T08:25:56.683 INFO:teuthology.orchestra.run.smithi170.stdout:Install 1 Package 2024-09-17T08:25:56.683 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:56.683 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 172 k 2024-09-17T08:25:56.683 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 494 k 2024-09-17T08:25:56.684 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2024-09-17T08:25:56.806 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : rbd-mirror-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:25:56.807 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:56.807 INFO:teuthology.orchestra.run.smithi118.stdout:Installed: 2024-09-17T08:25:56.807 INFO:teuthology.orchestra.run.smithi118.stdout: rbd-mirror-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:25:56.807 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:56.807 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2024-09-17T08:25:56.897 INFO:teuthology.orchestra.run.smithi170.stdout:rbd-nbd-18.2.4-855.g19e2a2be.el9.x86_64.rpm 807 kB/s | 172 kB 00:00 2024-09-17T08:25:56.897 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:25:56.897 INFO:teuthology.orchestra.run.smithi170.stdout:Total 801 kB/s | 172 kB 00:00 2024-09-17T08:25:56.898 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2024-09-17T08:25:56.905 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2024-09-17T08:25:56.905 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2024-09-17T08:25:56.970 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2024-09-17T08:25:56.970 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2024-09-17T08:25:57.108 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum -y install rbd-nbd 2024-09-17T08:25:57.122 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2024-09-17T08:25:57.219 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-nbd-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:25:57.655 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: rbd-nbd-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:25:57.707 INFO:teuthology.orchestra.run.smithi118.stdout:Last metadata expiration check: 0:01:40 ago on Tue 17 Sep 2024 08:24:17 AM UTC. 2024-09-17T08:25:57.840 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2024-09-17T08:25:57.841 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:25:57.841 INFO:teuthology.orchestra.run.smithi118.stdout: Package Architecture Version Repository Size 2024-09-17T08:25:57.841 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:25:57.841 INFO:teuthology.orchestra.run.smithi118.stdout:Installing: 2024-09-17T08:25:57.841 INFO:teuthology.orchestra.run.smithi118.stdout: rbd-nbd x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 172 k 2024-09-17T08:25:57.841 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:57.841 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction Summary 2024-09-17T08:25:57.841 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-09-17T08:25:57.841 INFO:teuthology.orchestra.run.smithi118.stdout:Install 1 Package 2024-09-17T08:25:57.842 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:57.842 INFO:teuthology.orchestra.run.smithi118.stdout:Total download size: 172 k 2024-09-17T08:25:57.842 INFO:teuthology.orchestra.run.smithi118.stdout:Installed size: 494 k 2024-09-17T08:25:57.842 INFO:teuthology.orchestra.run.smithi118.stdout:Downloading Packages: 2024-09-17T08:25:57.895 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-nbd-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:25:57.895 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:57.895 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2024-09-17T08:25:57.895 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-nbd-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:25:57.895 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:25:57.896 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-09-17T08:25:58.044 INFO:teuthology.orchestra.run.smithi118.stdout:rbd-nbd-18.2.4-855.g19e2a2be.el9.x86_64.rpm 851 kB/s | 172 kB 00:00 2024-09-17T08:25:58.044 INFO:teuthology.orchestra.run.smithi118.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:25:58.045 INFO:teuthology.orchestra.run.smithi118.stdout:Total 845 kB/s | 172 kB 00:00 2024-09-17T08:25:58.045 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction check 2024-09-17T08:25:58.052 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction check succeeded. 2024-09-17T08:25:58.053 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction test 2024-09-17T08:25:58.116 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction test succeeded. 2024-09-17T08:25:58.116 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction 2024-09-17T08:25:58.143 DEBUG:teuthology.parallel:result is None 2024-09-17T08:25:58.268 INFO:teuthology.orchestra.run.smithi118.stdout: Preparing : 1/1 2024-09-17T08:25:58.375 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : rbd-nbd-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:25:58.808 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: rbd-nbd-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:25:59.041 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : rbd-nbd-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:25:59.041 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:59.041 INFO:teuthology.orchestra.run.smithi118.stdout:Installed: 2024-09-17T08:25:59.041 INFO:teuthology.orchestra.run.smithi118.stdout: rbd-nbd-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:25:59.041 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:25:59.041 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2024-09-17T08:25:59.316 DEBUG:teuthology.parallel:result is None 2024-09-17T08:25:59.317 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-17T08:25:59.466 DEBUG:teuthology.orchestra.run.smithi118:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-17T08:25:59.488 INFO:teuthology.orchestra.run.smithi118.stdout:18.2.4-855.g19e2a2be.el9 2024-09-17T08:25:59.489 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-855.g19e2a2be.el9 2024-09-17T08:25:59.489 INFO:teuthology.task.install:The correct ceph version 18.2.4-855.g19e2a2be is installed. 2024-09-17T08:25:59.491 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-17T08:25:59.635 DEBUG:teuthology.orchestra.run.smithi170:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-17T08:25:59.656 INFO:teuthology.orchestra.run.smithi170.stdout:18.2.4-855.g19e2a2be.el9 2024-09-17T08:25:59.656 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-855.g19e2a2be.el9 2024-09-17T08:25:59.656 INFO:teuthology.task.install:The correct ceph version 18.2.4-855.g19e2a2be is installed. 2024-09-17T08:25:59.658 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-17T08:25:59.658 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-17T08:25:59.658 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T08:25:59.684 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-17T08:25:59.684 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T08:25:59.723 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-17T08:25:59.723 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-17T08:25:59.724 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T08:25:59.749 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T08:25:59.814 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-17T08:25:59.814 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T08:25:59.840 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T08:25:59.905 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-17T08:25:59.905 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-17T08:25:59.905 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T08:25:59.930 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T08:25:59.995 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-17T08:25:59.995 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T08:26:00.020 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T08:26:00.084 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-17T08:26:00.084 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-17T08:26:00.084 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T08:26:00.109 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T08:26:00.174 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-17T08:26:00.174 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T08:26:00.198 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T08:26:00.263 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-17T08:26:00.359 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'global': {'mon election default strategy': 1, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9', 'cephadm_mode': 'cephadm-package'} 2024-09-17T08:26:00.359 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-17T08:26:00.360 INFO:tasks.cephadm:Cluster fsid is 7862d192-74ce-11ef-bceb-c7b262605968 2024-09-17T08:26:00.360 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-17T08:26:00.360 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.118', 'mon.c': '[v2:172.21.15.118:3301,v1:172.21.15.118:6790]', 'mon.b': '172.21.15.170'} 2024-09-17T08:26:00.360 INFO:tasks.cephadm:First mon is mon.a on smithi118 2024-09-17T08:26:00.360 INFO:tasks.cephadm:First mgr is y 2024-09-17T08:26:00.360 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-17T08:26:00.361 DEBUG:teuthology.orchestra.run.smithi118:> sudo hostname $(hostname -s) 2024-09-17T08:26:00.386 DEBUG:teuthology.orchestra.run.smithi170:> sudo hostname $(hostname -s) 2024-09-17T08:26:00.421 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 on all hosts... 2024-09-17T08:26:00.421 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 pull 2024-09-17T08:26:00.428 DEBUG:teuthology.orchestra.run.smithi170:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 pull 2024-09-17T08:26:01.903 INFO:teuthology.orchestra.run.smithi170.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9... 2024-09-17T08:26:02.252 INFO:teuthology.orchestra.run.smithi118.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9... 2024-09-17T08:26:29.741 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-17T08:26:29.741 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_version": "ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable)", 2024-09-17T08:26:29.741 INFO:teuthology.orchestra.run.smithi170.stdout: "image_id": "980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89", 2024-09-17T08:26:29.741 INFO:teuthology.orchestra.run.smithi170.stdout: "repo_digests": [ 2024-09-17T08:26:29.742 INFO:teuthology.orchestra.run.smithi170.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9" 2024-09-17T08:26:29.742 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-17T08:26:29.742 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-17T08:26:33.734 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-17T08:26:33.734 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_version": "ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable)", 2024-09-17T08:26:33.735 INFO:teuthology.orchestra.run.smithi118.stdout: "image_id": "980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89", 2024-09-17T08:26:33.735 INFO:teuthology.orchestra.run.smithi118.stdout: "repo_digests": [ 2024-09-17T08:26:33.735 INFO:teuthology.orchestra.run.smithi118.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9" 2024-09-17T08:26:33.735 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-17T08:26:33.735 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-17T08:26:33.753 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /etc/ceph 2024-09-17T08:26:33.820 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /etc/ceph 2024-09-17T08:26:33.853 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 777 /etc/ceph 2024-09-17T08:26:33.878 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 777 /etc/ceph 2024-09-17T08:26:33.921 INFO:tasks.cephadm:Writing seed config... 2024-09-17T08:26:33.922 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-17T08:26:33.922 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-17T08:26:33.923 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-09-17T08:26:33.923 INFO:tasks.cephadm: override: [global] ms type = async 2024-09-17T08:26:33.923 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-17T08:26:33.923 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-17T08:26:33.923 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-17T08:26:33.923 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-17T08:26:33.923 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-17T08:26:33.923 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-17T08:26:33.924 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-17T08:26:33.924 DEBUG:teuthology.orchestra.run.smithi118:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-17T08:26:33.939 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 7862d192-74ce-11ef-bceb-c7b262605968 mon election default strategy = 1 ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-09-17T08:26:33.939 DEBUG:teuthology.orchestra.run.smithi118:mon.a> sudo journalctl -f -n 0 -u ceph-7862d192-74ce-11ef-bceb-c7b262605968@mon.a.service 2024-09-17T08:26:33.982 DEBUG:teuthology.orchestra.run.smithi118:mgr.y> sudo journalctl -f -n 0 -u ceph-7862d192-74ce-11ef-bceb-c7b262605968@mgr.y.service 2024-09-17T08:26:34.025 INFO:tasks.cephadm:Bootstrapping... 2024-09-17T08:26:34.025 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 -v bootstrap --fsid 7862d192-74ce-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.118 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-17T08:26:34.177 INFO:teuthology.orchestra.run.smithi118.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:26:34.177 INFO:teuthology.orchestra.run.smithi118.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9', '-v', 'bootstrap', '--fsid', '7862d192-74ce-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.118', '--skip-admin-label'] 2024-09-17T08:26:34.201 INFO:teuthology.orchestra.run.smithi118.stdout:/bin/podman: stdout 5.2.2 2024-09-17T08:26:34.202 INFO:teuthology.orchestra.run.smithi118.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-17T08:26:34.202 INFO:teuthology.orchestra.run.smithi118.stdout:Verifying podman|docker is present... 2024-09-17T08:26:34.224 INFO:teuthology.orchestra.run.smithi118.stdout:/bin/podman: stdout 5.2.2 2024-09-17T08:26:34.224 INFO:teuthology.orchestra.run.smithi118.stdout:Verifying lvm2 is present... 2024-09-17T08:26:34.224 INFO:teuthology.orchestra.run.smithi118.stdout:Verifying time synchronization is in place... 2024-09-17T08:26:34.231 INFO:teuthology.orchestra.run.smithi118.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-17T08:26:34.232 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-17T08:26:34.238 INFO:teuthology.orchestra.run.smithi118.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-17T08:26:34.238 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stdout inactive 2024-09-17T08:26:34.244 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stdout enabled 2024-09-17T08:26:34.251 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stdout active 2024-09-17T08:26:34.251 INFO:teuthology.orchestra.run.smithi118.stdout:Unit chronyd.service is enabled and running 2024-09-17T08:26:34.251 INFO:teuthology.orchestra.run.smithi118.stdout:Repeating the final host check... 2024-09-17T08:26:34.272 INFO:teuthology.orchestra.run.smithi118.stdout:/bin/podman: stdout 5.2.2 2024-09-17T08:26:34.273 INFO:teuthology.orchestra.run.smithi118.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-17T08:26:34.273 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl is present 2024-09-17T08:26:34.273 INFO:teuthology.orchestra.run.smithi118.stdout:lvcreate is present 2024-09-17T08:26:34.279 INFO:teuthology.orchestra.run.smithi118.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-17T08:26:34.279 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-17T08:26:34.285 INFO:teuthology.orchestra.run.smithi118.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-17T08:26:34.285 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stdout inactive 2024-09-17T08:26:34.292 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stdout enabled 2024-09-17T08:26:34.298 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stdout active 2024-09-17T08:26:34.298 INFO:teuthology.orchestra.run.smithi118.stdout:Unit chronyd.service is enabled and running 2024-09-17T08:26:34.298 INFO:teuthology.orchestra.run.smithi118.stdout:Host looks OK 2024-09-17T08:26:34.299 INFO:teuthology.orchestra.run.smithi118.stdout:Cluster fsid: 7862d192-74ce-11ef-bceb-c7b262605968 2024-09-17T08:26:34.299 INFO:teuthology.orchestra.run.smithi118.stdout:Acquiring lock 140549657902672 on /run/cephadm/7862d192-74ce-11ef-bceb-c7b262605968.lock 2024-09-17T08:26:34.299 INFO:teuthology.orchestra.run.smithi118.stdout:Lock 140549657902672 acquired on /run/cephadm/7862d192-74ce-11ef-bceb-c7b262605968.lock 2024-09-17T08:26:34.299 INFO:teuthology.orchestra.run.smithi118.stdout:Verifying IP 172.21.15.118 port 3300 ... 2024-09-17T08:26:34.299 INFO:teuthology.orchestra.run.smithi118.stdout:Verifying IP 172.21.15.118 port 6789 ... 2024-09-17T08:26:34.300 INFO:teuthology.orchestra.run.smithi118.stdout:Base mon IP(s) is [172.21.15.118:3300, 172.21.15.118:6789], mon addrv is [v2:172.21.15.118:3300,v1:172.21.15.118:6789] 2024-09-17T08:26:34.302 INFO:teuthology.orchestra.run.smithi118.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.118 metric 100 2024-09-17T08:26:34.302 INFO:teuthology.orchestra.run.smithi118.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.118 metric 100 2024-09-17T08:26:34.305 INFO:teuthology.orchestra.run.smithi118.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-17T08:26:34.305 INFO:teuthology.orchestra.run.smithi118.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-17T08:26:34.305 INFO:teuthology.orchestra.run.smithi118.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-17T08:26:34.307 INFO:teuthology.orchestra.run.smithi118.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-17T08:26:34.307 INFO:teuthology.orchestra.run.smithi118.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-17T08:26:34.307 INFO:teuthology.orchestra.run.smithi118.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-17T08:26:34.307 INFO:teuthology.orchestra.run.smithi118.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-17T08:26:34.308 INFO:teuthology.orchestra.run.smithi118.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d725/64 scope link noprefixroute 2024-09-17T08:26:34.308 INFO:teuthology.orchestra.run.smithi118.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-17T08:26:34.308 INFO:teuthology.orchestra.run.smithi118.stdout:Mon IP `172.21.15.118` is in CIDR network `172.21.0.0/20` 2024-09-17T08:26:34.308 INFO:teuthology.orchestra.run.smithi118.stdout:Mon IP `172.21.15.118` is in CIDR network `172.21.0.0/20` 2024-09-17T08:26:34.308 INFO:teuthology.orchestra.run.smithi118.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-17T08:26:34.309 INFO:teuthology.orchestra.run.smithi118.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-17T08:26:34.309 INFO:teuthology.orchestra.run.smithi118.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9... 2024-09-17T08:26:34.754 INFO:teuthology.orchestra.run.smithi118.stdout:/bin/podman: stdout 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 2024-09-17T08:26:34.754 INFO:teuthology.orchestra.run.smithi118.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9... 2024-09-17T08:26:34.754 INFO:teuthology.orchestra.run.smithi118.stdout:/bin/podman: stderr Getting image source signatures 2024-09-17T08:26:34.754 INFO:teuthology.orchestra.run.smithi118.stdout:/bin/podman: stderr Copying blob sha256:d8a707c21dc06a64315160cb2d8e524fa91943a344731b2a930ee3a716bebbdd 2024-09-17T08:26:34.754 INFO:teuthology.orchestra.run.smithi118.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-17T08:26:34.754 INFO:teuthology.orchestra.run.smithi118.stdout:/bin/podman: stderr Copying config sha256:980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 2024-09-17T08:26:34.754 INFO:teuthology.orchestra.run.smithi118.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-17T08:26:35.784 INFO:teuthology.orchestra.run.smithi118.stdout:ceph: stdout ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable) 2024-09-17T08:26:35.784 INFO:teuthology.orchestra.run.smithi118.stdout:Ceph version: ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable) 2024-09-17T08:26:35.784 INFO:teuthology.orchestra.run.smithi118.stdout:Extracting ceph user uid/gid from container image... 2024-09-17T08:26:36.841 INFO:teuthology.orchestra.run.smithi118.stdout:stat: stdout 167 167 2024-09-17T08:26:36.841 INFO:teuthology.orchestra.run.smithi118.stdout:Creating initial keys... 2024-09-17T08:26:37.714 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph-authtool: stdout AQA9PelmrwV6CxAAu86yP3N1YbIrvETaiXPS9g== 2024-09-17T08:26:38.628 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph-authtool: stdout AQA+Pelm5S9fAxAAvPDiQKf58jKzHsBpbT1zRw== 2024-09-17T08:26:39.608 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph-authtool: stdout AQA+Pelmu2hOOxAAXh7oN9t3ukF9M4Mmr9NejQ== 2024-09-17T08:26:39.609 INFO:teuthology.orchestra.run.smithi118.stdout:Creating initial monmap... 2024-09-17T08:26:40.574 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-17T08:26:40.574 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-09-17T08:26:40.574 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 7862d192-74ce-11ef-bceb-c7b262605968 2024-09-17T08:26:40.574 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-17T08:26:40.574 INFO:teuthology.orchestra.run.smithi118.stdout:monmaptool for a [v2:172.21.15.118:3300,v1:172.21.15.118:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-17T08:26:40.574 INFO:teuthology.orchestra.run.smithi118.stdout:setting min_mon_release = pacific 2024-09-17T08:26:40.574 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/monmaptool: set fsid to 7862d192-74ce-11ef-bceb-c7b262605968 2024-09-17T08:26:40.575 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-17T08:26:40.575 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:26:40.575 INFO:teuthology.orchestra.run.smithi118.stdout:Creating mon... 2024-09-17T08:26:41.472 INFO:teuthology.orchestra.run.smithi118.stdout:create mon.a on 2024-09-17T08:26:41.672 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-17T08:26:41.816 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-17T08:26:41.984 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-7862d192-74ce-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-7862d192-74ce-11ef-bceb-c7b262605968.target. 2024-09-17T08:26:41.985 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-7862d192-74ce-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-7862d192-74ce-11ef-bceb-c7b262605968.target. 2024-09-17T08:26:42.208 INFO:teuthology.orchestra.run.smithi118.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-7862d192-74ce-11ef-bceb-c7b262605968@mon.a 2024-09-17T08:26:42.208 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stderr Failed to reset failed state of unit ceph-7862d192-74ce-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-7862d192-74ce-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-17T08:26:42.369 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-7862d192-74ce-11ef-bceb-c7b262605968.target.wants/ceph-7862d192-74ce-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-7862d192-74ce-11ef-bceb-c7b262605968@.service. 2024-09-17T08:26:42.867 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:42 smithi118 podman[28982]: 2024-09-17 08:26:42.829209669 +0000 UTC m=+0.341447251 container start db01ec18c0b7f9bda4c47c7dac019409dd2355fd80714113ebce70dc97628cc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True) 2024-09-17T08:26:42.913 INFO:teuthology.orchestra.run.smithi118.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-17T08:26:42.913 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stdout disabled 2024-09-17T08:26:42.920 INFO:teuthology.orchestra.run.smithi118.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-17T08:26:42.920 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stdout inactive 2024-09-17T08:26:42.920 INFO:teuthology.orchestra.run.smithi118.stdout:firewalld.service is not enabled 2024-09-17T08:26:42.920 INFO:teuthology.orchestra.run.smithi118.stdout:Not possible to enable service . firewalld.service is not available 2024-09-17T08:26:42.920 INFO:teuthology.orchestra.run.smithi118.stdout:Waiting for mon to start... 2024-09-17T08:26:42.921 INFO:teuthology.orchestra.run.smithi118.stdout:Waiting for mon... 2024-09-17T08:26:43.128 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:42 smithi118 ceph-mon[29020]: mkfs 7862d192-74ce-11ef-bceb-c7b262605968 2024-09-17T08:26:43.128 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:42 smithi118 ceph-mon[29020]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-17T08:26:43.128 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:42 smithi118 bash[28982]: db01ec18c0b7f9bda4c47c7dac019409dd2355fd80714113ebce70dc97628cc2 2024-09-17T08:26:43.128 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:42 smithi118 systemd[1]: Started Ceph mon.a for 7862d192-74ce-11ef-bceb-c7b262605968. 2024-09-17T08:26:44.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:43 smithi118 ceph-mon[29020]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-17T08:26:44.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:43 smithi118 ceph-mon[29020]: monmap e1: 1 mons at {a=[v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T08:26:44.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:43 smithi118 ceph-mon[29020]: fsmap 2024-09-17T08:26:44.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:43 smithi118 ceph-mon[29020]: osdmap e1: 0 total, 0 up, 0 in 2024-09-17T08:26:44.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:43 smithi118 ceph-mon[29020]: mgrmap e1: no daemons active 2024-09-17T08:26:44.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:43 smithi118 ceph-mon[29020]: from='client.? 172.21.15.118:0/125951801' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-17T08:26:44.035 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout cluster: 2024-09-17T08:26:44.035 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout id: 7862d192-74ce-11ef-bceb-c7b262605968 2024-09-17T08:26:44.035 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-17T08:26:44.035 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout 2024-09-17T08:26:44.035 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout services: 2024-09-17T08:26:44.035 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.558005s) 2024-09-17T08:26:44.035 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-17T08:26:44.036 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-17T08:26:44.036 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout 2024-09-17T08:26:44.036 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout data: 2024-09-17T08:26:44.036 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-17T08:26:44.036 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-17T08:26:44.036 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-17T08:26:44.036 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout pgs: 2024-09-17T08:26:44.036 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout 2024-09-17T08:26:44.036 INFO:teuthology.orchestra.run.smithi118.stdout:mon is available 2024-09-17T08:26:44.036 INFO:teuthology.orchestra.run.smithi118.stdout:Assimilating anything we can from ceph.conf... 2024-09-17T08:26:45.105 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:44 smithi118 ceph-mon[29020]: from='client.? 172.21.15.118:0/2564900609' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-17T08:26:45.105 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:44 smithi118 ceph-mon[29020]: from='client.? 172.21.15.118:0/2564900609' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-17T08:26:45.109 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout 2024-09-17T08:26:45.109 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout [global] 2024-09-17T08:26:45.109 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout fsid = 7862d192-74ce-11ef-bceb-c7b262605968 2024-09-17T08:26:45.109 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.118:3300,v1:172.21.15.118:6789] 2024-09-17T08:26:45.109 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-17T08:26:45.109 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-17T08:26:45.110 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-17T08:26:45.110 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-17T08:26:45.110 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout 2024-09-17T08:26:45.110 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-17T08:26:45.110 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-17T08:26:45.110 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout 2024-09-17T08:26:45.110 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout [osd] 2024-09-17T08:26:45.110 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-17T08:26:45.110 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-17T08:26:45.110 INFO:teuthology.orchestra.run.smithi118.stdout:Generating new minimal ceph.conf... 2024-09-17T08:26:46.113 INFO:teuthology.orchestra.run.smithi118.stdout:Restarting the monitor... 2024-09-17T08:26:46.128 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:45 smithi118 ceph-mon[29020]: from='client.? 172.21.15.118:0/765948622' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:26:46.384 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:46 smithi118 systemd[1]: Stopping Ceph mon.a for 7862d192-74ce-11ef-bceb-c7b262605968... 2024-09-17T08:26:46.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:46 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mon-a[28997]: 2024-09-17T08:26:46.383+0000 7fa33e780640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:26:46.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:46 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mon-a[28997]: 2024-09-17T08:26:46.383+0000 7fa33e780640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-17T08:26:46.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:46 smithi118 podman[29340]: 2024-09-17 08:26:46.557274594 +0000 UTC m=+0.286200856 container died db01ec18c0b7f9bda4c47c7dac019409dd2355fd80714113ebce70dc97628cc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mon-a, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-17T08:26:47.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:46 smithi118 podman[29340]: 2024-09-17 08:26:46.905883974 +0000 UTC m=+0.634810228 container cleanup db01ec18c0b7f9bda4c47c7dac019409dd2355fd80714113ebce70dc97628cc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mon-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=reef-19e2a2b, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-17T08:26:47.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:46 smithi118 bash[29340]: ceph-7862d192-74ce-11ef-bceb-c7b262605968-mon-a 2024-09-17T08:26:47.518 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 podman[29357]: 2024-09-17 08:26:47.173617216 +0000 UTC m=+0.610848537 container remove db01ec18c0b7f9bda4c47c7dac019409dd2355fd80714113ebce70dc97628cc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-17T08:26:47.518 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 systemd[1]: ceph-7862d192-74ce-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-17T08:26:47.518 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 systemd[1]: Stopped Ceph mon.a for 7862d192-74ce-11ef-bceb-c7b262605968. 2024-09-17T08:26:47.518 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 systemd[1]: Starting Ceph mon.a for 7862d192-74ce-11ef-bceb-c7b262605968... 2024-09-17T08:26:47.769 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 podman[29453]: 2024-09-17 08:26:47.421123991 +0000 UTC m=+0.015465228 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-17T08:26:47.770 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 podman[29453]: 2024-09-17 08:26:47.524311343 +0000 UTC m=+0.118652580 container create f167927aab3b4be72e05bb035c2d678ab85691374dc81d4a9bfd58cdc6f04b26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-17T08:26:47.770 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 podman[29453]: 2024-09-17 08:26:47.735051358 +0000 UTC m=+0.329392596 container init f167927aab3b4be72e05bb035c2d678ab85691374dc81d4a9bfd58cdc6f04b26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T08:26:47.770 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 podman[29453]: 2024-09-17 08:26:47.739886911 +0000 UTC m=+0.334228140 container start f167927aab3b4be72e05bb035c2d678ab85691374dc81d4a9bfd58cdc6f04b26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mon-a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-17T08:26:47.770 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: set uid:gid to 167:167 (ceph:ceph) 2024-09-17T08:26:47.770 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable), process ceph-mon, pid 7 2024-09-17T08:26:47.770 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: pidfile_write: ignore empty --pid-file 2024-09-17T08:26:47.770 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: load: jerasure load: lrc 2024-09-17T08:26:47.770 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: RocksDB version: 7.9.2 2024-09-17T08:26:47.770 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Git sha 0 2024-09-17T08:26:47.770 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Compile date 2024-09-16 15:15:13 2024-09-17T08:26:47.770 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: DB SUMMARY 2024-09-17T08:26:47.770 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: DB Session ID: NEWH42GFPT3Y7C9UQ6XJ 2024-09-17T08:26:47.771 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: CURRENT file: CURRENT 2024-09-17T08:26:47.771 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: IDENTITY file: IDENTITY 2024-09-17T08:26:47.771 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-17T08:26:47.771 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-17T08:26:47.771 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 84158 ; 2024-09-17T08:26:47.771 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.error_if_exists: 0 2024-09-17T08:26:47.771 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.create_if_missing: 0 2024-09-17T08:26:47.771 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.paranoid_checks: 1 2024-09-17T08:26:47.771 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-17T08:26:47.771 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-17T08:26:47.771 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-17T08:26:47.771 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.env: 0x56206c9e4bc0 2024-09-17T08:26:47.771 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.fs: PosixFileSystem 2024-09-17T08:26:47.772 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.info_log: 0x56206f038be0 2024-09-17T08:26:47.772 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.max_file_opening_threads: 16 2024-09-17T08:26:47.772 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.statistics: (nil) 2024-09-17T08:26:47.772 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.use_fsync: 0 2024-09-17T08:26:47.772 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.max_log_file_size: 0 2024-09-17T08:26:47.772 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-17T08:26:47.772 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-17T08:26:47.772 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.keep_log_file_num: 1000 2024-09-17T08:26:47.772 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.recycle_log_file_num: 0 2024-09-17T08:26:47.772 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.allow_fallocate: 1 2024-09-17T08:26:47.772 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.allow_mmap_reads: 0 2024-09-17T08:26:47.772 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.allow_mmap_writes: 0 2024-09-17T08:26:47.772 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.use_direct_reads: 0 2024-09-17T08:26:47.773 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-17T08:26:47.773 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.create_missing_column_families: 0 2024-09-17T08:26:47.773 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.db_log_dir: 2024-09-17T08:26:47.773 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.wal_dir: 2024-09-17T08:26:47.773 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-17T08:26:47.773 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-17T08:26:47.773 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-17T08:26:47.773 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-17T08:26:47.773 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-17T08:26:47.773 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-17T08:26:47.773 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.advise_random_on_open: 1 2024-09-17T08:26:47.773 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.db_write_buffer_size: 0 2024-09-17T08:26:47.773 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.write_buffer_manager: 0x56206f046320 2024-09-17T08:26:47.773 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-17T08:26:47.773 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-17T08:26:47.774 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-17T08:26:47.774 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.rate_limiter: (nil) 2024-09-17T08:26:47.774 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-17T08:26:47.774 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.wal_recovery_mode: 2 2024-09-17T08:26:47.775 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.enable_thread_tracking: 0 2024-09-17T08:26:47.775 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.enable_pipelined_write: 0 2024-09-17T08:26:47.775 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.unordered_write: 0 2024-09-17T08:26:47.775 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-17T08:26:47.775 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-17T08:26:47.775 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-17T08:26:47.775 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-17T08:26:47.775 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.row_cache: None 2024-09-17T08:26:47.776 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.wal_filter: None 2024-09-17T08:26:47.776 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-17T08:26:47.776 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.allow_ingest_behind: 0 2024-09-17T08:26:47.776 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.two_write_queues: 0 2024-09-17T08:26:47.776 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.manual_wal_flush: 0 2024-09-17T08:26:47.776 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.wal_compression: 0 2024-09-17T08:26:47.776 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.atomic_flush: 0 2024-09-17T08:26:47.776 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-17T08:26:47.776 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-17T08:26:47.776 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-17T08:26:47.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.log_readahead_size: 0 2024-09-17T08:26:47.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-17T08:26:47.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.best_efforts_recovery: 0 2024-09-17T08:26:47.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-17T08:26:47.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-17T08:26:47.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.allow_data_in_errors: 0 2024-09-17T08:26:47.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.db_host_id: __hostname__ 2024-09-17T08:26:47.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-17T08:26:47.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.max_background_jobs: 2 2024-09-17T08:26:47.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.max_background_compactions: -1 2024-09-17T08:26:47.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.max_subcompactions: 1 2024-09-17T08:26:47.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-17T08:26:47.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-17T08:26:47.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-17T08:26:47.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.max_total_wal_size: 0 2024-09-17T08:26:47.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-17T08:26:47.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-17T08:26:47.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-17T08:26:47.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-17T08:26:47.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.max_open_files: -1 2024-09-17T08:26:47.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.bytes_per_sync: 0 2024-09-17T08:26:47.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-17T08:26:47.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-17T08:26:47.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.compaction_readahead_size: 0 2024-09-17T08:26:47.780 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.max_background_flushes: -1 2024-09-17T08:26:47.780 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Compression algorithms supported: 2024-09-17T08:26:47.780 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: kZSTD supported: 0 2024-09-17T08:26:47.780 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: kXpressCompression supported: 0 2024-09-17T08:26:47.780 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: kBZip2Compression supported: 0 2024-09-17T08:26:47.780 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-17T08:26:47.780 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: kLZ4Compression supported: 1 2024-09-17T08:26:47.780 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: kZlibCompression supported: 1 2024-09-17T08:26:47.781 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: kLZ4HCCompression supported: 1 2024-09-17T08:26:47.781 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: kSnappyCompression supported: 1 2024-09-17T08:26:47.781 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-17T08:26:47.781 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-17T08:26:47.781 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-17T08:26:47.781 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-17T08:26:47.781 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-17T08:26:47.781 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.merge_operator: 2024-09-17T08:26:47.781 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.compaction_filter: None 2024-09-17T08:26:47.781 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.compaction_filter_factory: None 2024-09-17T08:26:47.781 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.sst_partitioner_factory: None 2024-09-17T08:26:47.782 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-17T08:26:47.782 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-17T08:26:47.782 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56206f0387e0) 2024-09-17T08:26:47.782 INFO:journalctl@ceph.mon.a.smithi118.stdout: cache_index_and_filter_blocks: 1 2024-09-17T08:26:47.782 INFO:journalctl@ceph.mon.a.smithi118.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-17T08:26:47.782 INFO:journalctl@ceph.mon.a.smithi118.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-17T08:26:47.782 INFO:journalctl@ceph.mon.a.smithi118.stdout: pin_top_level_index_and_filter: 1 2024-09-17T08:26:47.782 INFO:journalctl@ceph.mon.a.smithi118.stdout: index_type: 0 2024-09-17T08:26:47.782 INFO:journalctl@ceph.mon.a.smithi118.stdout: data_block_index_type: 0 2024-09-17T08:26:47.782 INFO:journalctl@ceph.mon.a.smithi118.stdout: index_shortening: 1 2024-09-17T08:26:47.782 INFO:journalctl@ceph.mon.a.smithi118.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-17T08:26:47.782 INFO:journalctl@ceph.mon.a.smithi118.stdout: checksum: 4 2024-09-17T08:26:47.782 INFO:journalctl@ceph.mon.a.smithi118.stdout: no_block_cache: 0 2024-09-17T08:26:47.782 INFO:journalctl@ceph.mon.a.smithi118.stdout: block_cache: 0x56206f02d350 2024-09-17T08:26:47.783 INFO:journalctl@ceph.mon.a.smithi118.stdout: block_cache_name: BinnedLRUCache 2024-09-17T08:26:47.783 INFO:journalctl@ceph.mon.a.smithi118.stdout: block_cache_options: 2024-09-17T08:26:47.783 INFO:journalctl@ceph.mon.a.smithi118.stdout: capacity : 536870912 2024-09-17T08:26:47.783 INFO:journalctl@ceph.mon.a.smithi118.stdout: num_shard_bits : 4 2024-09-17T08:26:47.783 INFO:journalctl@ceph.mon.a.smithi118.stdout: strict_capacity_limit : 0 2024-09-17T08:26:47.783 INFO:journalctl@ceph.mon.a.smithi118.stdout: high_pri_pool_ratio: 0.000 2024-09-17T08:26:47.783 INFO:journalctl@ceph.mon.a.smithi118.stdout: block_cache_compressed: (nil) 2024-09-17T08:26:47.783 INFO:journalctl@ceph.mon.a.smithi118.stdout: persistent_cache: (nil) 2024-09-17T08:26:47.783 INFO:journalctl@ceph.mon.a.smithi118.stdout: block_size: 4096 2024-09-17T08:26:47.783 INFO:journalctl@ceph.mon.a.smithi118.stdout: block_size_deviation: 10 2024-09-17T08:26:47.783 INFO:journalctl@ceph.mon.a.smithi118.stdout: block_restart_interval: 16 2024-09-17T08:26:47.783 INFO:journalctl@ceph.mon.a.smithi118.stdout: index_block_restart_interval: 1 2024-09-17T08:26:47.783 INFO:journalctl@ceph.mon.a.smithi118.stdout: metadata_block_size: 4096 2024-09-17T08:26:47.783 INFO:journalctl@ceph.mon.a.smithi118.stdout: partition_filters: 0 2024-09-17T08:26:47.783 INFO:journalctl@ceph.mon.a.smithi118.stdout: use_delta_encoding: 1 2024-09-17T08:26:47.784 INFO:journalctl@ceph.mon.a.smithi118.stdout: filter_policy: bloomfilter 2024-09-17T08:26:47.784 INFO:journalctl@ceph.mon.a.smithi118.stdout: whole_key_filtering: 1 2024-09-17T08:26:47.784 INFO:journalctl@ceph.mon.a.smithi118.stdout: verify_compression: 0 2024-09-17T08:26:47.784 INFO:journalctl@ceph.mon.a.smithi118.stdout: read_amp_bytes_per_bit: 0 2024-09-17T08:26:47.784 INFO:journalctl@ceph.mon.a.smithi118.stdout: format_version: 5 2024-09-17T08:26:47.784 INFO:journalctl@ceph.mon.a.smithi118.stdout: enable_index_compression: 1 2024-09-17T08:26:47.784 INFO:journalctl@ceph.mon.a.smithi118.stdout: block_align: 0 2024-09-17T08:26:47.784 INFO:journalctl@ceph.mon.a.smithi118.stdout: max_auto_readahead_size: 262144 2024-09-17T08:26:47.784 INFO:journalctl@ceph.mon.a.smithi118.stdout: prepopulate_block_cache: 0 2024-09-17T08:26:47.784 INFO:journalctl@ceph.mon.a.smithi118.stdout: initial_auto_readahead_size: 8192 2024-09-17T08:26:47.784 INFO:journalctl@ceph.mon.a.smithi118.stdout: num_file_reads_for_auto_readahead: 2 2024-09-17T08:26:47.784 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.write_buffer_size: 33554432 2024-09-17T08:26:47.784 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.max_write_buffer_number: 2 2024-09-17T08:26:47.784 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.compression: NoCompression 2024-09-17T08:26:47.785 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.bottommost_compression: Disabled 2024-09-17T08:26:47.785 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.prefix_extractor: nullptr 2024-09-17T08:26:47.785 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-17T08:26:47.785 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.num_levels: 7 2024-09-17T08:26:47.785 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-17T08:26:47.785 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-17T08:26:47.785 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-17T08:26:47.785 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-17T08:26:47.785 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-17T08:26:47.785 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-17T08:26:47.785 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-17T08:26:47.785 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-17T08:26:47.785 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-17T08:26:47.785 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-17T08:26:47.786 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-17T08:26:47.786 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-17T08:26:47.786 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-17T08:26:47.786 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.compression_opts.level: 32767 2024-09-17T08:26:47.786 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.compression_opts.strategy: 0 2024-09-17T08:26:47.786 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-17T08:26:47.786 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-17T08:26:47.786 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-17T08:26:47.786 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-17T08:26:47.786 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.compression_opts.enabled: false 2024-09-17T08:26:47.786 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-17T08:26:47.786 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-17T08:26:47.786 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-17T08:26:47.786 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-17T08:26:47.786 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.target_file_size_base: 67108864 2024-09-17T08:26:47.787 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-17T08:26:47.787 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-17T08:26:47.787 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-17T08:26:47.787 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-17T08:26:47.787 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-17T08:26:47.787 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-17T08:26:47.787 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-17T08:26:47.787 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-17T08:26:47.787 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-17T08:26:47.787 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-17T08:26:47.787 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-17T08:26:47.787 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-17T08:26:47.787 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-17T08:26:47.787 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-17T08:26:47.787 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.arena_block_size: 1048576 2024-09-17T08:26:47.788 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-17T08:26:47.788 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-17T08:26:47.788 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.disable_auto_compactions: 0 2024-09-17T08:26:47.788 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-17T08:26:47.788 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-17T08:26:47.788 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-17T08:26:47.788 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-17T08:26:47.788 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-17T08:26:47.789 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-17T08:26:47.789 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-17T08:26:47.789 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-17T08:26:47.789 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-17T08:26:47.789 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-17T08:26:47.789 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-17T08:26:47.789 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.inplace_update_support: 0 2024-09-17T08:26:47.789 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-17T08:26:47.789 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-17T08:26:47.789 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-17T08:26:47.790 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-17T08:26:47.790 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.bloom_locality: 0 2024-09-17T08:26:47.790 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.max_successive_merges: 0 2024-09-17T08:26:47.790 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-17T08:26:47.790 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.paranoid_file_checks: 0 2024-09-17T08:26:47.790 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.force_consistency_checks: 1 2024-09-17T08:26:47.790 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.report_bg_io_stats: 0 2024-09-17T08:26:47.790 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.ttl: 2592000 2024-09-17T08:26:47.790 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-17T08:26:47.791 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-17T08:26:47.791 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-17T08:26:47.791 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.enable_blob_files: false 2024-09-17T08:26:47.791 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.min_blob_size: 0 2024-09-17T08:26:47.791 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.blob_file_size: 268435456 2024-09-17T08:26:47.791 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-17T08:26:47.791 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-17T08:26:47.791 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-17T08:26:47.791 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-17T08:26:47.791 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-17T08:26:47.791 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.blob_file_starting_level: 0 2024-09-17T08:26:47.791 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-17T08:26:47.791 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-17T08:26:47.791 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-17T08:26:47.792 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ae02515a-cc68-456b-9cd9-5968c4fb1c24 2024-09-17T08:26:47.792 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726561607765895, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-17T08:26:47.792 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-17T08:26:47.792 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726561607766724, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 80844, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 252, "table_properties": {"data_size": 79005, "index_size": 223, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 9995, "raw_average_key_size": 47, "raw_value_size": 73311, "raw_average_value_size": 347, "num_data_blocks": 10, "num_entries": 211, "num_filter_entries": 211, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726561607, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ae02515a-cc68-456b-9cd9-5968c4fb1c24", "db_session_id": "NEWH42GFPT3Y7C9UQ6XJ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-17T08:26:47.792 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726561607766818, "job": 1, "event": "recovery_finished"} 2024-09-17T08:26:47.792 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-17T08:26:47.792 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-17T08:26:47.792 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56206f058e00 2024-09-17T08:26:47.792 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: DB pointer 0x56206f14c000 2024-09-17T08:26:47.792 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-17T08:26:47.792 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-17T08:26:47.792 INFO:journalctl@ceph.mon.a.smithi118.stdout: ** DB Stats ** 2024-09-17T08:26:47.792 INFO:journalctl@ceph.mon.a.smithi118.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-17T08:26:47.792 INFO:journalctl@ceph.mon.a.smithi118.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-17T08:26:47.793 INFO:journalctl@ceph.mon.a.smithi118.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-17T08:26:47.793 INFO:journalctl@ceph.mon.a.smithi118.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-17T08:26:47.793 INFO:journalctl@ceph.mon.a.smithi118.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-17T08:26:47.793 INFO:journalctl@ceph.mon.a.smithi118.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-17T08:26:47.793 INFO:journalctl@ceph.mon.a.smithi118.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-17T08:26:47.793 INFO:journalctl@ceph.mon.a.smithi118.stdout: 2024-09-17T08:26:47.793 INFO:journalctl@ceph.mon.a.smithi118.stdout: ** Compaction Stats [default] ** 2024-09-17T08:26:47.793 INFO:journalctl@ceph.mon.a.smithi118.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-17T08:26:47.793 INFO:journalctl@ceph.mon.a.smithi118.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-17T08:26:47.793 INFO:journalctl@ceph.mon.a.smithi118.stdout: L0 2/0 80.81 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 143.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-17T08:26:47.793 INFO:journalctl@ceph.mon.a.smithi118.stdout: Sum 2/0 80.81 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 143.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-17T08:26:47.793 INFO:journalctl@ceph.mon.a.smithi118.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 143.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-17T08:26:47.793 INFO:journalctl@ceph.mon.a.smithi118.stdout: 2024-09-17T08:26:47.793 INFO:journalctl@ceph.mon.a.smithi118.stdout: ** Compaction Stats [default] ** 2024-09-17T08:26:47.794 INFO:journalctl@ceph.mon.a.smithi118.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-17T08:26:47.794 INFO:journalctl@ceph.mon.a.smithi118.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-17T08:26:47.794 INFO:journalctl@ceph.mon.a.smithi118.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 143.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-17T08:26:47.794 INFO:journalctl@ceph.mon.a.smithi118.stdout: 2024-09-17T08:26:47.794 INFO:journalctl@ceph.mon.a.smithi118.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-17T08:26:47.794 INFO:journalctl@ceph.mon.a.smithi118.stdout: 2024-09-17T08:26:47.794 INFO:journalctl@ceph.mon.a.smithi118.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-17T08:26:47.794 INFO:journalctl@ceph.mon.a.smithi118.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-17T08:26:47.794 INFO:journalctl@ceph.mon.a.smithi118.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-17T08:26:47.795 INFO:journalctl@ceph.mon.a.smithi118.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-17T08:26:47.795 INFO:journalctl@ceph.mon.a.smithi118.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-17T08:26:47.795 INFO:journalctl@ceph.mon.a.smithi118.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-17T08:26:47.795 INFO:journalctl@ceph.mon.a.smithi118.stdout: Cumulative compaction: 0.00 GB write, 26.32 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-17T08:26:47.795 INFO:journalctl@ceph.mon.a.smithi118.stdout: Interval compaction: 0.00 GB write, 26.32 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-17T08:26:47.795 INFO:journalctl@ceph.mon.a.smithi118.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-17T08:26:47.795 INFO:journalctl@ceph.mon.a.smithi118.stdout: Block cache BinnedLRUCache@0x56206f02d350#7 capacity: 512.00 MB usage: 1.17 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-17T08:26:47.795 INFO:journalctl@ceph.mon.a.smithi118.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.41 KB,7.7486e-05%) Misc(1,0.00 KB,0%) 2024-09-17T08:26:47.795 INFO:journalctl@ceph.mon.a.smithi118.stdout: 2024-09-17T08:26:47.795 INFO:journalctl@ceph.mon.a.smithi118.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-17T08:26:47.795 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: starting mon.a rank 0 at public addrs [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] at bind addrs [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 7862d192-74ce-11ef-bceb-c7b262605968 2024-09-17T08:26:47.795 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: mon.a@-1(???) e1 preinit fsid 7862d192-74ce-11ef-bceb-c7b262605968 2024-09-17T08:26:47.795 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: mon.a@-1(???).mds e1 new map 2024-09-17T08:26:47.795 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: mon.a@-1(???).mds e1 print_map 2024-09-17T08:26:47.796 INFO:journalctl@ceph.mon.a.smithi118.stdout: e1 2024-09-17T08:26:47.796 INFO:journalctl@ceph.mon.a.smithi118.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T08:26:47.796 INFO:journalctl@ceph.mon.a.smithi118.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T08:26:47.796 INFO:journalctl@ceph.mon.a.smithi118.stdout: legacy client fscid: -1 2024-09-17T08:26:47.796 INFO:journalctl@ceph.mon.a.smithi118.stdout: 2024-09-17T08:26:47.796 INFO:journalctl@ceph.mon.a.smithi118.stdout: No filesystems configured 2024-09-17T08:26:47.796 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-17T08:26:47.796 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T08:26:47.796 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T08:26:47.796 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T08:26:47.796 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-17T08:26:47.831 INFO:teuthology.orchestra.run.smithi118.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-17T08:26:48.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-17T08:26:48.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: monmap e1: 1 mons at {a=[v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T08:26:48.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: fsmap 2024-09-17T08:26:48.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: osdmap e1: 0 total, 0 up, 0 in 2024-09-17T08:26:48.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 ceph-mon[29491]: mgrmap e1: no daemons active 2024-09-17T08:26:48.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 bash[29453]: f167927aab3b4be72e05bb035c2d678ab85691374dc81d4a9bfd58cdc6f04b26 2024-09-17T08:26:48.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:47 smithi118 systemd[1]: Started Ceph mon.a for 7862d192-74ce-11ef-bceb-c7b262605968. 2024-09-17T08:26:48.952 INFO:teuthology.orchestra.run.smithi118.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-17T08:26:48.974 INFO:teuthology.orchestra.run.smithi118.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-17T08:26:48.975 INFO:teuthology.orchestra.run.smithi118.stdout:Creating mgr... 2024-09-17T08:26:48.975 INFO:teuthology.orchestra.run.smithi118.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-17T08:26:48.975 INFO:teuthology.orchestra.run.smithi118.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-17T08:26:49.215 INFO:teuthology.orchestra.run.smithi118.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-7862d192-74ce-11ef-bceb-c7b262605968@mgr.y 2024-09-17T08:26:49.215 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stderr Failed to reset failed state of unit ceph-7862d192-74ce-11ef-bceb-c7b262605968@mgr.y.service: Unit ceph-7862d192-74ce-11ef-bceb-c7b262605968@mgr.y.service not loaded. 2024-09-17T08:26:49.360 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-7862d192-74ce-11ef-bceb-c7b262605968.target.wants/ceph-7862d192-74ce-11ef-bceb-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-7862d192-74ce-11ef-bceb-c7b262605968@.service. 2024-09-17T08:26:49.707 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:49 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/1398224927' entity='client.admin' 2024-09-17T08:26:49.708 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:26:49 smithi118 podman[29728]: 2024-09-17 08:26:49.593903927 +0000 UTC m=+0.115100002 container create e3931710a4691a3fbe1ae831c8db36f36640f304601d01196f4efa46cbceca09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-17T08:26:49.862 INFO:teuthology.orchestra.run.smithi118.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-17T08:26:49.862 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stdout disabled 2024-09-17T08:26:49.870 INFO:teuthology.orchestra.run.smithi118.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-17T08:26:49.870 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stdout inactive 2024-09-17T08:26:49.870 INFO:teuthology.orchestra.run.smithi118.stdout:firewalld.service is not enabled 2024-09-17T08:26:49.870 INFO:teuthology.orchestra.run.smithi118.stdout:Not possible to enable service . firewalld.service is not available 2024-09-17T08:26:49.878 INFO:teuthology.orchestra.run.smithi118.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-17T08:26:49.878 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stdout disabled 2024-09-17T08:26:49.886 INFO:teuthology.orchestra.run.smithi118.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-17T08:26:49.886 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stdout inactive 2024-09-17T08:26:49.886 INFO:teuthology.orchestra.run.smithi118.stdout:firewalld.service is not enabled 2024-09-17T08:26:49.886 INFO:teuthology.orchestra.run.smithi118.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-17T08:26:49.886 INFO:teuthology.orchestra.run.smithi118.stdout:Waiting for mgr to start... 2024-09-17T08:26:49.886 INFO:teuthology.orchestra.run.smithi118.stdout:Waiting for mgr... 2024-09-17T08:26:50.132 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:26:49 smithi118 podman[29728]: 2024-09-17 08:26:49.788520392 +0000 UTC m=+0.309716477 container init e3931710a4691a3fbe1ae831c8db36f36640f304601d01196f4efa46cbceca09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-17T08:26:50.132 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:26:49 smithi118 podman[29728]: 2024-09-17 08:26:49.793223399 +0000 UTC m=+0.314419475 container start e3931710a4691a3fbe1ae831c8db36f36640f304601d01196f4efa46cbceca09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y, GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-17T08:26:50.132 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:26:49 smithi118 bash[29728]: e3931710a4691a3fbe1ae831c8db36f36640f304601d01196f4efa46cbceca09 2024-09-17T08:26:50.132 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:26:49 smithi118 systemd[1]: Started Ceph mgr.y for 7862d192-74ce-11ef-bceb-c7b262605968. 2024-09-17T08:26:50.132 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:26:49 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:26:49.923+0000 7fd760347040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T08:26:50.132 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:26:49 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:26:49.980+0000 7fd760347040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T08:26:50.399 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:26:50 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:26:50.132+0000 7fd760347040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T08:26:50.399 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:26:50 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:26:50.204+0000 7fd760347040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T08:26:50.399 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:26:50 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:26:50.353+0000 7fd760347040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:26:50.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:50 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3861921585' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-17T08:26:50.777 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:26:50 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:26:50.502+0000 7fd760347040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T08:26:51.112 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:26:50 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:26:50.814+0000 7fd760347040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T08:26:51.112 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:26:50 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:26:50.876+0000 7fd760347040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T08:26:51.112 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:26:51 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:26:51.059+0000 7fd760347040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T08:26:51.116 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout 2024-09-17T08:26:51.116 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout { 2024-09-17T08:26:51.116 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "fsid": "7862d192-74ce-11ef-bceb-c7b262605968", 2024-09-17T08:26:51.117 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "health": { 2024-09-17T08:26:51.117 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-17T08:26:51.117 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-17T08:26:51.117 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-17T08:26:51.117 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:26:51.117 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-17T08:26:51.117 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-17T08:26:51.117 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout 0 2024-09-17T08:26:51.117 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout ], 2024-09-17T08:26:51.117 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-17T08:26:51.117 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "a" 2024-09-17T08:26:51.117 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout ], 2024-09-17T08:26:51.117 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-17T08:26:51.118 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-17T08:26:51.118 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T08:26:51.118 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-17T08:26:51.118 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-17T08:26:51.118 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:26:51.118 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-17T08:26:51.118 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T08:26:51.118 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-17T08:26:51.118 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-17T08:26:51.118 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-17T08:26:51.118 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-17T08:26:51.122 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-17T08:26:51.122 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-17T08:26:51.122 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:26:51.122 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-17T08:26:51.122 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-17T08:26:51.122 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-17T08:26:51.122 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-17T08:26:51.122 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-17T08:26:51.122 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-17T08:26:51.123 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-17T08:26:51.123 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-17T08:26:51.123 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-17T08:26:51.123 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:26:51.123 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-17T08:26:51.123 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T08:26:51.123 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-17T08:26:51.123 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-17T08:26:51.123 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:26:51.123 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-17T08:26:51.123 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-17T08:26:51.123 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-17T08:26:51.123 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-17T08:26:51.124 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-17T08:26:51.124 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-17T08:26:51.124 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "restful" 2024-09-17T08:26:51.124 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout ], 2024-09-17T08:26:51.124 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-17T08:26:51.124 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:26:51.124 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-17T08:26:51.124 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T08:26:51.124 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "modified": "2024-09-17T08:26:42.865812+0000", 2024-09-17T08:26:51.124 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-17T08:26:51.124 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:26:51.125 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-17T08:26:51.125 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout } 2024-09-17T08:26:51.125 INFO:teuthology.orchestra.run.smithi118.stdout:mgr not available, waiting (1/15)... 2024-09-17T08:26:52.277 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:26:51 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:26:51.919+0000 7fd760347040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T08:26:52.277 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:26:51 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:26:51.985+0000 7fd760347040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T08:26:52.948 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:26:52 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:26:52.693+0000 7fd760347040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T08:26:52.948 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:26:52 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:26:52.807+0000 7fd760347040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T08:26:52.948 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:26:52 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:26:52.947+0000 7fd760347040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:26:53.225 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:26:53 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:26:53.008+0000 7fd760347040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T08:26:53.226 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:26:53 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:26:53.157+0000 7fd760347040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T08:26:53.226 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:26:53 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:26:53.225+0000 7fd760347040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T08:26:53.522 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:26:53 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:26:53.286+0000 7fd760347040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T08:26:53.522 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:26:53 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:26:53.452+0000 7fd760347040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T08:26:53.522 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:26:53 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:26:53.521+0000 7fd760347040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T08:26:53.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:53 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3676235259' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-17T08:26:53.777 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:26:53 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:26:53.589+0000 7fd760347040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T08:26:53.777 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:26:53 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:26:53.676+0000 7fd760347040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T08:26:53.777 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:26:53 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:26:53.735+0000 7fd760347040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T08:26:54.168 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:26:53 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:26:53.794+0000 7fd760347040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T08:26:54.189 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout 2024-09-17T08:26:54.189 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout { 2024-09-17T08:26:54.189 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "fsid": "7862d192-74ce-11ef-bceb-c7b262605968", 2024-09-17T08:26:54.189 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "health": { 2024-09-17T08:26:54.190 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-17T08:26:54.190 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-17T08:26:54.190 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-17T08:26:54.190 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:26:54.190 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-17T08:26:54.190 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-17T08:26:54.190 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout 0 2024-09-17T08:26:54.190 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout ], 2024-09-17T08:26:54.190 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-17T08:26:54.190 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "a" 2024-09-17T08:26:54.190 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout ], 2024-09-17T08:26:54.190 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-17T08:26:54.190 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-17T08:26:54.190 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T08:26:54.191 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-17T08:26:54.191 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-17T08:26:54.191 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:26:54.191 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-17T08:26:54.191 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T08:26:54.191 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-17T08:26:54.191 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-17T08:26:54.191 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-17T08:26:54.191 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-17T08:26:54.191 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-17T08:26:54.191 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-17T08:26:54.191 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:26:54.191 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-17T08:26:54.191 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-17T08:26:54.192 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-17T08:26:54.192 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-17T08:26:54.192 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-17T08:26:54.192 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-17T08:26:54.192 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-17T08:26:54.192 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-17T08:26:54.192 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-17T08:26:54.192 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:26:54.192 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-17T08:26:54.192 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T08:26:54.192 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-17T08:26:54.192 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-17T08:26:54.192 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:26:54.192 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-17T08:26:54.193 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-17T08:26:54.193 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-17T08:26:54.193 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-17T08:26:54.193 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-17T08:26:54.193 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-17T08:26:54.193 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "restful" 2024-09-17T08:26:54.193 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout ], 2024-09-17T08:26:54.193 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-17T08:26:54.193 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:26:54.193 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-17T08:26:54.193 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T08:26:54.193 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "modified": "2024-09-17T08:26:42.865812+0000", 2024-09-17T08:26:54.193 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-17T08:26:54.193 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:26:54.193 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-17T08:26:54.194 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout } 2024-09-17T08:26:54.194 INFO:teuthology.orchestra.run.smithi118.stdout:mgr not available, waiting (2/15)... 2024-09-17T08:26:55.027 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:26:54 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:26:54.577+0000 7fd760347040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T08:26:55.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:54 smithi118 ceph-mon[29491]: Activating manager daemon y 2024-09-17T08:26:55.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:54 smithi118 ceph-mon[29491]: mgrmap e2: y(active, starting, since 0.00313178s) 2024-09-17T08:26:55.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:54 smithi118 ceph-mon[29491]: from='mgr.14100 172.21.15.118:0/1569226416' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-17T08:26:55.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:54 smithi118 ceph-mon[29491]: from='mgr.14100 172.21.15.118:0/1569226416' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-17T08:26:55.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:54 smithi118 ceph-mon[29491]: from='mgr.14100 172.21.15.118:0/1569226416' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-17T08:26:55.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:54 smithi118 ceph-mon[29491]: from='mgr.14100 172.21.15.118:0/1569226416' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:26:55.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:54 smithi118 ceph-mon[29491]: from='mgr.14100 172.21.15.118:0/1569226416' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-17T08:26:55.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:54 smithi118 ceph-mon[29491]: Manager daemon y is now available 2024-09-17T08:26:55.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:54 smithi118 ceph-mon[29491]: from='mgr.14100 172.21.15.118:0/1569226416' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-17T08:26:55.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:54 smithi118 ceph-mon[29491]: from='mgr.14100 172.21.15.118:0/1569226416' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-17T08:26:55.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:54 smithi118 ceph-mon[29491]: from='mgr.14100 172.21.15.118:0/1569226416' entity='mgr.y' 2024-09-17T08:26:55.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:54 smithi118 ceph-mon[29491]: from='mgr.14100 172.21.15.118:0/1569226416' entity='mgr.y' 2024-09-17T08:26:55.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:54 smithi118 ceph-mon[29491]: from='mgr.14100 172.21.15.118:0/1569226416' entity='mgr.y' 2024-09-17T08:26:56.867 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:56 smithi118 ceph-mon[29491]: mgrmap e3: y(active, since 1.00661s) 2024-09-17T08:26:57.508 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout 2024-09-17T08:26:57.508 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout { 2024-09-17T08:26:57.508 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "fsid": "7862d192-74ce-11ef-bceb-c7b262605968", 2024-09-17T08:26:57.509 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "health": { 2024-09-17T08:26:57.509 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-17T08:26:57.509 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-17T08:26:57.509 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-17T08:26:57.509 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:26:57.509 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-17T08:26:57.509 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-17T08:26:57.509 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout 0 2024-09-17T08:26:57.509 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout ], 2024-09-17T08:26:57.509 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-17T08:26:57.509 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "a" 2024-09-17T08:26:57.509 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout ], 2024-09-17T08:26:57.509 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-17T08:26:57.510 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-17T08:26:57.510 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T08:26:57.510 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-17T08:26:57.510 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-17T08:26:57.510 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:26:57.510 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-17T08:26:57.510 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T08:26:57.510 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-17T08:26:57.510 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-17T08:26:57.510 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-17T08:26:57.510 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-17T08:26:57.510 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-17T08:26:57.510 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-17T08:26:57.511 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:26:57.511 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-17T08:26:57.511 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-17T08:26:57.511 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-17T08:26:57.511 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-17T08:26:57.511 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-17T08:26:57.511 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-17T08:26:57.511 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-17T08:26:57.511 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-17T08:26:57.511 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-17T08:26:57.511 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:26:57.512 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-17T08:26:57.512 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T08:26:57.512 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-17T08:26:57.512 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-17T08:26:57.512 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:26:57.512 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-17T08:26:57.512 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-17T08:26:57.513 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-17T08:26:57.513 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-17T08:26:57.513 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-17T08:26:57.513 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-17T08:26:57.513 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "restful" 2024-09-17T08:26:57.513 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout ], 2024-09-17T08:26:57.513 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-17T08:26:57.513 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:26:57.513 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-17T08:26:57.514 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T08:26:57.514 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "modified": "2024-09-17T08:26:42.865812+0000", 2024-09-17T08:26:57.514 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-17T08:26:57.514 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:26:57.514 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-17T08:26:57.514 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout } 2024-09-17T08:26:57.514 INFO:teuthology.orchestra.run.smithi118.stdout:mgr is available 2024-09-17T08:26:57.752 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:57 smithi118 ceph-mon[29491]: mgrmap e4: y(active, since 2s) 2024-09-17T08:26:57.752 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:57 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3432345240' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-17T08:26:59.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:26:58 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/4155952903' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-17T08:27:05.207 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout 2024-09-17T08:27:05.207 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout [global] 2024-09-17T08:27:05.207 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout fsid = 7862d192-74ce-11ef-bceb-c7b262605968 2024-09-17T08:27:05.207 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-17T08:27:05.207 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-17T08:27:05.208 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-17T08:27:05.208 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-17T08:27:05.208 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout 2024-09-17T08:27:05.208 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-17T08:27:05.208 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-17T08:27:05.208 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout 2024-09-17T08:27:05.208 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout [osd] 2024-09-17T08:27:05.208 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-17T08:27:05.208 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-17T08:27:05.208 INFO:teuthology.orchestra.run.smithi118.stdout:Enabling cephadm module... 2024-09-17T08:27:08.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:08 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3049114242' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-17T08:27:08.925 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:08 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ignoring --setuser ceph since I am not root 2024-09-17T08:27:08.925 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:08 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ignoring --setgroup ceph since I am not root 2024-09-17T08:27:08.925 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:08 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:08.722+0000 7fe2dabfd040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T08:27:08.925 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:08 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:08.777+0000 7fe2dabfd040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T08:27:08.925 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:08 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:08.923+0000 7fe2dabfd040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T08:27:09.270 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:08 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:08.988+0000 7fe2dabfd040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T08:27:09.270 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:09 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:09.131+0000 7fe2dabfd040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:27:09.270 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:09 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:09.269+0000 7fe2dabfd040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T08:27:10.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:09 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3049114242' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-17T08:27:10.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:09 smithi118 ceph-mon[29491]: mgrmap e5: y(active, since 14s) 2024-09-17T08:27:10.027 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:09 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:09.573+0000 7fe2dabfd040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T08:27:10.027 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:09 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:09.633+0000 7fe2dabfd040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T08:27:10.027 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:09 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:09.813+0000 7fe2dabfd040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T08:27:10.945 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:10.678+0000 7fe2dabfd040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T08:27:10.946 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:10.744+0000 7fe2dabfd040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T08:27:11.518 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:11 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:11.464+0000 7fe2dabfd040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T08:27:11.910 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:11 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:11.585+0000 7fe2dabfd040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T08:27:11.910 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:11 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:11.730+0000 7fe2dabfd040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:27:11.910 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:11 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:11.791+0000 7fe2dabfd040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T08:27:12.218 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:11 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3052167532' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-17T08:27:12.219 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:11 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:11.945+0000 7fe2dabfd040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T08:27:12.219 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:12 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:12.014+0000 7fe2dabfd040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T08:27:12.219 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:12 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:12.074+0000 7fe2dabfd040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T08:27:12.504 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:12 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:12.232+0000 7fe2dabfd040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T08:27:12.504 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:12 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:12.296+0000 7fe2dabfd040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T08:27:12.504 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:12 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:12.361+0000 7fe2dabfd040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T08:27:12.504 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:12 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:12.445+0000 7fe2dabfd040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T08:27:12.504 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:12 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:12.503+0000 7fe2dabfd040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T08:27:12.540 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout { 2024-09-17T08:27:12.540 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-17T08:27:12.540 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-17T08:27:12.541 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-17T08:27:12.541 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-17T08:27:12.541 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout } 2024-09-17T08:27:12.541 INFO:teuthology.orchestra.run.smithi118.stdout:Waiting for the mgr to restart... 2024-09-17T08:27:12.541 INFO:teuthology.orchestra.run.smithi118.stdout:Waiting for mgr epoch 5... 2024-09-17T08:27:12.777 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:12 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:12.565+0000 7fe2dabfd040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T08:27:13.777 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:13 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:13.368+0000 7fe2dabfd040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T08:27:13.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:13 smithi118 ceph-mon[29491]: Active manager daemon y restarted 2024-09-17T08:27:13.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:13 smithi118 ceph-mon[29491]: Activating manager daemon y 2024-09-17T08:27:13.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:13 smithi118 ceph-mon[29491]: osdmap e2: 0 total, 0 up, 0 in 2024-09-17T08:27:13.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:13 smithi118 ceph-mon[29491]: mgrmap e6: y(active, starting, since 0.00300098s) 2024-09-17T08:27:13.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:13 smithi118 ceph-mon[29491]: from='mgr.14118 172.21.15.118:0/3912074562' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:27:13.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:13 smithi118 ceph-mon[29491]: from='mgr.14118 172.21.15.118:0/3912074562' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-17T08:27:13.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:13 smithi118 ceph-mon[29491]: from='mgr.14118 172.21.15.118:0/3912074562' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-17T08:27:13.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:13 smithi118 ceph-mon[29491]: from='mgr.14118 172.21.15.118:0/3912074562' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-17T08:27:13.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:13 smithi118 ceph-mon[29491]: from='mgr.14118 172.21.15.118:0/3912074562' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-17T08:27:13.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:13 smithi118 ceph-mon[29491]: Manager daemon y is now available 2024-09-17T08:27:13.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:13 smithi118 ceph-mon[29491]: from='mgr.14118 172.21.15.118:0/3912074562' entity='mgr.y' 2024-09-17T08:27:13.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:13 smithi118 ceph-mon[29491]: from='mgr.14118 172.21.15.118:0/3912074562' entity='mgr.y' 2024-09-17T08:27:13.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:13 smithi118 ceph-mon[29491]: from='mgr.14118 172.21.15.118:0/3912074562' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:27:13.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:13 smithi118 ceph-mon[29491]: from='mgr.14118 172.21.15.118:0/3912074562' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:27:13.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:13 smithi118 ceph-mon[29491]: from='mgr.14118 172.21.15.118:0/3912074562' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-17T08:27:13.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:13 smithi118 ceph-mon[29491]: from='mgr.14118 172.21.15.118:0/3912074562' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-17T08:27:14.683 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:14 smithi118 ceph-mon[29491]: Found migration_current of "None". Setting to last migration. 2024-09-17T08:27:14.683 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:14 smithi118 ceph-mon[29491]: from='mgr.14118 172.21.15.118:0/3912074562' entity='mgr.y' 2024-09-17T08:27:14.683 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:14 smithi118 ceph-mon[29491]: from='mgr.14118 172.21.15.118:0/3912074562' entity='mgr.y' 2024-09-17T08:27:14.684 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:14 smithi118 ceph-mon[29491]: mgrmap e7: y(active, since 1.00471s) 2024-09-17T08:27:14.953 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout { 2024-09-17T08:27:14.953 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-17T08:27:14.953 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-17T08:27:14.953 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout } 2024-09-17T08:27:14.953 INFO:teuthology.orchestra.run.smithi118.stdout:mgr epoch 5 is available 2024-09-17T08:27:14.953 INFO:teuthology.orchestra.run.smithi118.stdout:Setting orchestrator backend to cephadm... 2024-09-17T08:27:15.616 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:15 smithi118 ceph-mon[29491]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-17T08:27:15.617 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:15 smithi118 ceph-mon[29491]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-17T08:27:15.617 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:15 smithi118 ceph-mon[29491]: from='mgr.14118 172.21.15.118:0/3912074562' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:27:15.617 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:15 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:15.317+0000 7fe2428dd640 -1 log_channel(cephadm) log [ERR] : [17/Sep/2024:08:27:15] ENGINE Error in HTTPServer.serve 2024-09-17T08:27:15.617 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:15 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: Traceback (most recent call last): 2024-09-17T08:27:15.617 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:15 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T08:27:15.617 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:15 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: self._connections.run(self.expiration_interval) 2024-09-17T08:27:15.617 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:15 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T08:27:15.617 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:15 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: self._run(expiration_interval) 2024-09-17T08:27:15.618 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:15 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T08:27:15.618 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:15 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T08:27:15.618 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:15 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T08:27:15.618 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:15 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T08:27:15.618 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:15 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T08:27:15.618 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:15 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: s = self.context.wrap_socket( 2024-09-17T08:27:15.618 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:15 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T08:27:15.618 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:15 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: return self.sslsocket_class._create( 2024-09-17T08:27:15.618 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:15 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T08:27:15.618 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:15 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: self.do_handshake() 2024-09-17T08:27:15.618 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:15 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T08:27:15.618 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:15 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: self._sslobj.do_handshake() 2024-09-17T08:27:15.618 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:15 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T08:27:15.618 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:15 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:16.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:16 smithi118 ceph-mon[29491]: [17/Sep/2024:08:27:15] ENGINE Bus STARTING 2024-09-17T08:27:16.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:16 smithi118 ceph-mon[29491]: [17/Sep/2024:08:27:15] ENGINE Serving on http://172.21.15.118:8765 2024-09-17T08:27:16.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:16 smithi118 ceph-mon[29491]: [17/Sep/2024:08:27:15] ENGINE Serving on https://172.21.15.118:7150 2024-09-17T08:27:16.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:16 smithi118 ceph-mon[29491]: [17/Sep/2024:08:27:15] ENGINE Bus STARTED 2024-09-17T08:27:16.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:16 smithi118 ceph-mon[29491]: [17/Sep/2024:08:27:15] ENGINE Error in HTTPServer.serve 2024-09-17T08:27:16.527 INFO:journalctl@ceph.mon.a.smithi118.stdout: Traceback (most recent call last): 2024-09-17T08:27:16.528 INFO:journalctl@ceph.mon.a.smithi118.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T08:27:16.528 INFO:journalctl@ceph.mon.a.smithi118.stdout: self._connections.run(self.expiration_interval) 2024-09-17T08:27:16.528 INFO:journalctl@ceph.mon.a.smithi118.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T08:27:16.528 INFO:journalctl@ceph.mon.a.smithi118.stdout: self._run(expiration_interval) 2024-09-17T08:27:16.528 INFO:journalctl@ceph.mon.a.smithi118.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T08:27:16.528 INFO:journalctl@ceph.mon.a.smithi118.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T08:27:16.528 INFO:journalctl@ceph.mon.a.smithi118.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T08:27:16.528 INFO:journalctl@ceph.mon.a.smithi118.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T08:27:16.528 INFO:journalctl@ceph.mon.a.smithi118.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T08:27:16.528 INFO:journalctl@ceph.mon.a.smithi118.stdout: s = self.context.wrap_socket( 2024-09-17T08:27:16.528 INFO:journalctl@ceph.mon.a.smithi118.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T08:27:16.528 INFO:journalctl@ceph.mon.a.smithi118.stdout: return self.sslsocket_class._create( 2024-09-17T08:27:16.528 INFO:journalctl@ceph.mon.a.smithi118.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T08:27:16.529 INFO:journalctl@ceph.mon.a.smithi118.stdout: self.do_handshake() 2024-09-17T08:27:16.529 INFO:journalctl@ceph.mon.a.smithi118.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T08:27:16.529 INFO:journalctl@ceph.mon.a.smithi118.stdout: self._sslobj.do_handshake() 2024-09-17T08:27:16.529 INFO:journalctl@ceph.mon.a.smithi118.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T08:27:16.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:16 smithi118 ceph-mon[29491]: mgrmap e8: y(active, since 2s) 2024-09-17T08:27:16.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:16 smithi118 ceph-mon[29491]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:27:16.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:16 smithi118 ceph-mon[29491]: from='mgr.14118 172.21.15.118:0/3912074562' entity='mgr.y' 2024-09-17T08:27:16.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:16 smithi118 ceph-mon[29491]: from='mgr.14118 172.21.15.118:0/3912074562' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:27:17.423 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-17T08:27:17.423 INFO:teuthology.orchestra.run.smithi118.stdout:Generating ssh key... 2024-09-17T08:27:18.014 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:17 smithi118 ceph-mon[29491]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:27:18.527 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:18 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: Generating public/private rsa key pair. 2024-09-17T08:27:18.527 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:18 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: Your identification has been saved in /tmp/tmpjhkkmv_n/key 2024-09-17T08:27:18.527 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:18 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: Your public key has been saved in /tmp/tmpjhkkmv_n/key.pub 2024-09-17T08:27:18.527 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:18 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: The key fingerprint is: 2024-09-17T08:27:18.528 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:18 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: SHA256:QJdDayyVJMonRxdgF729HvQbg7SlBeLCGqO0xWBA6NU ceph-7862d192-74ce-11ef-bceb-c7b262605968 2024-09-17T08:27:18.528 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:18 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: The key's randomart image is: 2024-09-17T08:27:18.528 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:18 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: +---[RSA 3072]----+ 2024-09-17T08:27:18.528 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:18 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: | oo.. *+X* | 2024-09-17T08:27:18.528 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:18 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: |. ooE B+.o . | 2024-09-17T08:27:18.528 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:18 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: |. ..+o=.+o + . | 2024-09-17T08:27:18.528 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:18 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: | . .+=+o o + o | 2024-09-17T08:27:18.528 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:18 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: | . + +S. o O | 2024-09-17T08:27:18.528 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:18 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: | o . * + | 2024-09-17T08:27:18.528 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:18 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: | . . + | 2024-09-17T08:27:18.528 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:18 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: | . . | 2024-09-17T08:27:18.528 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:18 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: | | 2024-09-17T08:27:18.528 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:18 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: +----[SHA256]-----+ 2024-09-17T08:27:19.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:19 smithi118 ceph-mon[29491]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:27:19.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:19 smithi118 ceph-mon[29491]: Generating ssh key... 2024-09-17T08:27:19.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:19 smithi118 ceph-mon[29491]: from='mgr.14118 172.21.15.118:0/3912074562' entity='mgr.y' 2024-09-17T08:27:19.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:19 smithi118 ceph-mon[29491]: from='mgr.14118 172.21.15.118:0/3912074562' entity='mgr.y' 2024-09-17T08:27:20.201 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2ktbReshp3Nm7MfczDYGo0FBIJA02uqHgM1hk+SIFOFIgH1oEot+6ADddCsklol3GcvYffRJgFsCzqURiVgMAhChVzAekuYGb8/KPNmKTStmLpAz96Og+03EW00F7vN/5wbFAH9o2y1iYOVRcw72fldO4KJzlUnLM0A9CR7a91fwDKZaqQiIuyOQPbMgoe9pH4E7FepnYnT9ZSF556HADZaF8xIuT49uneDQoXiJO8ssvERyK55rezLLNfVqFSq/UZLUblzMegMzbFjAsSlc1aTiwyBWoVDHvZpjlw1wYHlKLhHD18x8YZyG778gjPj3h6TZNs7zrqA+B4+TXeWfg5kYbrfn9U1ANVTi1NF61Izb5sICW+86boOmwGxaFgrq1a5o4WGat+EEGKaZFPnmSS5tRkoIUEu6Gruv+GTHcno1J2tWmhWmCuB3GPaGUCn1xou+z1fNXODX6342mBrBtAQLmjbGvxFFfjgQ619S1riSChDz923uz1dXfb5Bhbzc= ceph-7862d192-74ce-11ef-bceb-c7b262605968 2024-09-17T08:27:20.202 INFO:teuthology.orchestra.run.smithi118.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-17T08:27:20.202 INFO:teuthology.orchestra.run.smithi118.stdout:Adding key to root@localhost authorized_keys... 2024-09-17T08:27:20.206 INFO:teuthology.orchestra.run.smithi118.stdout:Adding host smithi118... 2024-09-17T08:27:20.910 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:20 smithi118 ceph-mon[29491]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:27:21.824 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:21 smithi118 ceph-mon[29491]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi118", "addr": "172.21.15.118", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:27:22.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:22 smithi118 ceph-mon[29491]: Deploying cephadm binary to smithi118 2024-09-17T08:27:24.075 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout Added host 'smithi118' with addr '172.21.15.118' 2024-09-17T08:27:24.075 INFO:teuthology.orchestra.run.smithi118.stdout:Deploying unmanaged mon service... 2024-09-17T08:27:24.735 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:24 smithi118 ceph-mon[29491]: from='mgr.14118 172.21.15.118:0/3912074562' entity='mgr.y' 2024-09-17T08:27:24.736 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:24 smithi118 ceph-mon[29491]: Added host smithi118 2024-09-17T08:27:24.736 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:24 smithi118 ceph-mon[29491]: from='mgr.14118 172.21.15.118:0/3912074562' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:27:26.184 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:25 smithi118 ceph-mon[29491]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:27:26.184 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:25 smithi118 ceph-mon[29491]: Saving service mon spec with placement count:5 2024-09-17T08:27:26.184 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:25 smithi118 ceph-mon[29491]: from='mgr.14118 172.21.15.118:0/3912074562' entity='mgr.y' 2024-09-17T08:27:26.185 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:25 smithi118 ceph-mon[29491]: from='mgr.14118 172.21.15.118:0/3912074562' entity='mgr.y' 2024-09-17T08:27:26.186 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-17T08:27:26.186 INFO:teuthology.orchestra.run.smithi118.stdout:Deploying unmanaged mgr service... 2024-09-17T08:27:27.344 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:27 smithi118 ceph-mon[29491]: from='mgr.14118 172.21.15.118:0/3912074562' entity='mgr.y' 2024-09-17T08:27:27.345 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:27 smithi118 ceph-mon[29491]: from='mgr.14118 172.21.15.118:0/3912074562' entity='mgr.y' 2024-09-17T08:27:28.013 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-17T08:27:28.267 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:28 smithi118 ceph-mon[29491]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:27:28.267 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:28 smithi118 ceph-mon[29491]: Saving service mgr spec with placement count:2 2024-09-17T08:27:29.769 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:29 smithi118 ceph-mon[29491]: from='mgr.14118 172.21.15.118:0/3912074562' entity='mgr.y' 2024-09-17T08:27:29.769 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:29 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/2051518911' entity='client.admin' 2024-09-17T08:27:30.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:30 smithi118 ceph-mon[29491]: from='mgr.14118 172.21.15.118:0/3912074562' entity='mgr.y' 2024-09-17T08:27:30.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:30 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/4208191333' entity='client.admin' 2024-09-17T08:27:31.756 INFO:teuthology.orchestra.run.smithi118.stdout:Enabling the dashboard module... 2024-09-17T08:27:32.532 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:32 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3801504404' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-17T08:27:32.862 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:32 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ignoring --setuser ceph since I am not root 2024-09-17T08:27:32.863 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:32 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ignoring --setgroup ceph since I am not root 2024-09-17T08:27:32.863 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:32 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:32.651+0000 7fa5f13a6040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T08:27:32.863 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:32 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:32.710+0000 7fa5f13a6040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T08:27:32.863 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:32 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:32.861+0000 7fa5f13a6040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T08:27:33.177 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:32 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:32.930+0000 7fa5f13a6040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T08:27:33.177 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:33 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:33.077+0000 7fa5f13a6040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:27:33.454 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:33 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:33.224+0000 7fa5f13a6040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T08:27:33.712 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:33 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3801504404' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-17T08:27:33.713 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:33 smithi118 ceph-mon[29491]: mgrmap e9: y(active, since 19s) 2024-09-17T08:27:33.713 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:33 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:33.556+0000 7fa5f13a6040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T08:27:33.713 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:33 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:33.618+0000 7fa5f13a6040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T08:27:34.027 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:33 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:33.820+0000 7fa5f13a6040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T08:27:34.536 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout { 2024-09-17T08:27:34.536 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-17T08:27:34.536 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-17T08:27:34.536 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-17T08:27:34.537 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-17T08:27:34.537 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout } 2024-09-17T08:27:34.537 INFO:teuthology.orchestra.run.smithi118.stdout:Waiting for the mgr to restart... 2024-09-17T08:27:34.537 INFO:teuthology.orchestra.run.smithi118.stdout:Waiting for mgr epoch 9... 2024-09-17T08:27:34.758 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:34 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/4084641432' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-17T08:27:34.759 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:34 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:34.712+0000 7fa5f13a6040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T08:27:35.027 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:34 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:34.782+0000 7fa5f13a6040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T08:27:35.748 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:35 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:35.492+0000 7fa5f13a6040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T08:27:35.748 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:35 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:35.607+0000 7fa5f13a6040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T08:27:36.020 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:35 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:35.748+0000 7fa5f13a6040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:27:36.021 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:35 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:35.808+0000 7fa5f13a6040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T08:27:36.021 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:35 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:35.954+0000 7fa5f13a6040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T08:27:36.021 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:36 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:36.021+0000 7fa5f13a6040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T08:27:36.277 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:36 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:36.080+0000 7fa5f13a6040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T08:27:36.277 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:36 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:36.235+0000 7fa5f13a6040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T08:27:36.560 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:36 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:36.299+0000 7fa5f13a6040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T08:27:36.560 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:36 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:36.365+0000 7fa5f13a6040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T08:27:36.561 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:36 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:36.446+0000 7fa5f13a6040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T08:27:36.561 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:36 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:36.503+0000 7fa5f13a6040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T08:27:36.561 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:36 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:36.559+0000 7fa5f13a6040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T08:27:37.777 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:37 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:37.327+0000 7fa5f13a6040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T08:27:37.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:37 smithi118 ceph-mon[29491]: Active manager daemon y restarted 2024-09-17T08:27:37.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:37 smithi118 ceph-mon[29491]: Activating manager daemon y 2024-09-17T08:27:37.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:37 smithi118 ceph-mon[29491]: osdmap e3: 0 total, 0 up, 0 in 2024-09-17T08:27:37.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:37 smithi118 ceph-mon[29491]: mgrmap e10: y(active, starting, since 0.00300533s) 2024-09-17T08:27:37.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:37 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:27:37.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:37 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-17T08:27:37.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:37 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-17T08:27:37.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:37 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-17T08:27:37.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:37 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-17T08:27:37.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:37 smithi118 ceph-mon[29491]: Manager daemon y is now available 2024-09-17T08:27:37.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:37 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:27:37.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:37 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-17T08:27:37.780 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:37 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-17T08:27:38.777 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:38 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:38.408+0000 7fa557830640 -1 log_channel(cephadm) log [ERR] : [17/Sep/2024:08:27:38] ENGINE Error in HTTPServer.serve 2024-09-17T08:27:38.778 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:38 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: Traceback (most recent call last): 2024-09-17T08:27:38.778 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:38 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T08:27:38.778 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:38 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: self._connections.run(self.expiration_interval) 2024-09-17T08:27:38.778 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:38 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T08:27:38.778 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:38 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: self._run(expiration_interval) 2024-09-17T08:27:38.778 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:38 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T08:27:38.778 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:38 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T08:27:38.778 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:38 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T08:27:38.778 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:38 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T08:27:38.779 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:38 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T08:27:38.779 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:38 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: s = self.context.wrap_socket( 2024-09-17T08:27:38.779 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:38 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T08:27:38.779 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:38 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: return self.sslsocket_class._create( 2024-09-17T08:27:38.779 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:38 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T08:27:38.779 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:38 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: self.do_handshake() 2024-09-17T08:27:38.779 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:38 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T08:27:38.779 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:38 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: self._sslobj.do_handshake() 2024-09-17T08:27:38.779 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:38 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T08:27:38.780 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:27:38 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:27:39.137 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout { 2024-09-17T08:27:39.137 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-17T08:27:39.137 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-17T08:27:39.137 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout } 2024-09-17T08:27:39.137 INFO:teuthology.orchestra.run.smithi118.stdout:mgr epoch 9 is available 2024-09-17T08:27:39.138 INFO:teuthology.orchestra.run.smithi118.stdout:Generating a dashboard self-signed certificate... 2024-09-17T08:27:39.387 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:39 smithi118 ceph-mon[29491]: [17/Sep/2024:08:27:38] ENGINE Bus STARTING 2024-09-17T08:27:39.387 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:39 smithi118 ceph-mon[29491]: [17/Sep/2024:08:27:38] ENGINE Serving on http://172.21.15.118:8765 2024-09-17T08:27:39.387 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:39 smithi118 ceph-mon[29491]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-17T08:27:39.387 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:39 smithi118 ceph-mon[29491]: mgrmap e11: y(active, since 1.00551s) 2024-09-17T08:27:39.387 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:39 smithi118 ceph-mon[29491]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-17T08:27:39.388 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:39 smithi118 ceph-mon[29491]: [17/Sep/2024:08:27:38] ENGINE Serving on https://172.21.15.118:7150 2024-09-17T08:27:39.388 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:39 smithi118 ceph-mon[29491]: [17/Sep/2024:08:27:38] ENGINE Bus STARTED 2024-09-17T08:27:39.388 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:39 smithi118 ceph-mon[29491]: [17/Sep/2024:08:27:38] ENGINE Error in HTTPServer.serve 2024-09-17T08:27:39.388 INFO:journalctl@ceph.mon.a.smithi118.stdout: Traceback (most recent call last): 2024-09-17T08:27:39.388 INFO:journalctl@ceph.mon.a.smithi118.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T08:27:39.388 INFO:journalctl@ceph.mon.a.smithi118.stdout: self._connections.run(self.expiration_interval) 2024-09-17T08:27:39.388 INFO:journalctl@ceph.mon.a.smithi118.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T08:27:39.388 INFO:journalctl@ceph.mon.a.smithi118.stdout: self._run(expiration_interval) 2024-09-17T08:27:39.388 INFO:journalctl@ceph.mon.a.smithi118.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T08:27:39.388 INFO:journalctl@ceph.mon.a.smithi118.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T08:27:39.388 INFO:journalctl@ceph.mon.a.smithi118.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T08:27:39.388 INFO:journalctl@ceph.mon.a.smithi118.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T08:27:39.388 INFO:journalctl@ceph.mon.a.smithi118.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T08:27:39.389 INFO:journalctl@ceph.mon.a.smithi118.stdout: s = self.context.wrap_socket( 2024-09-17T08:27:39.389 INFO:journalctl@ceph.mon.a.smithi118.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T08:27:39.389 INFO:journalctl@ceph.mon.a.smithi118.stdout: return self.sslsocket_class._create( 2024-09-17T08:27:39.389 INFO:journalctl@ceph.mon.a.smithi118.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T08:27:39.389 INFO:journalctl@ceph.mon.a.smithi118.stdout: self.do_handshake() 2024-09-17T08:27:39.389 INFO:journalctl@ceph.mon.a.smithi118.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T08:27:39.389 INFO:journalctl@ceph.mon.a.smithi118.stdout: self._sslobj.do_handshake() 2024-09-17T08:27:39.389 INFO:journalctl@ceph.mon.a.smithi118.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T08:27:40.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:40 smithi118 ceph-mon[29491]: mgrmap e12: y(active, since 2s) 2024-09-17T08:27:40.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:40 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:27:40.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:40 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:27:40.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:40 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:27:40.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:40 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:27:40.534 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-17T08:27:40.534 INFO:teuthology.orchestra.run.smithi118.stdout:Creating initial admin user... 2024-09-17T08:27:41.399 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:41 smithi118 ceph-mon[29491]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:27:42.209 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$wd6ictstvvhMhdu/o8ZZw.mdpTNvFwtL089ElvoN7PtCaEh0EuJbW", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726561661, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-17T08:27:42.209 INFO:teuthology.orchestra.run.smithi118.stdout:Fetching dashboard port number... 2024-09-17T08:27:42.463 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:42 smithi118 ceph-mon[29491]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:27:42.463 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:42 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:27:43.706 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:43 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/576967511' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-17T08:27:43.984 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout 8443 2024-09-17T08:27:43.994 INFO:teuthology.orchestra.run.smithi118.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-17T08:27:43.994 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stdout disabled 2024-09-17T08:27:44.001 INFO:teuthology.orchestra.run.smithi118.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-17T08:27:44.001 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stdout inactive 2024-09-17T08:27:44.001 INFO:teuthology.orchestra.run.smithi118.stdout:firewalld.service is not enabled 2024-09-17T08:27:44.001 INFO:teuthology.orchestra.run.smithi118.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-17T08:27:44.002 INFO:teuthology.orchestra.run.smithi118.stdout:Ceph Dashboard is now available at: 2024-09-17T08:27:44.002 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:27:44.003 INFO:teuthology.orchestra.run.smithi118.stdout: URL: https://smithi118.front.sepia.ceph.com:8443/ 2024-09-17T08:27:44.003 INFO:teuthology.orchestra.run.smithi118.stdout: User: admin 2024-09-17T08:27:44.003 INFO:teuthology.orchestra.run.smithi118.stdout: Password: vepzd5uns3 2024-09-17T08:27:44.003 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:27:44.003 INFO:teuthology.orchestra.run.smithi118.stdout:Saving cluster configuration to /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config directory 2024-09-17T08:27:44.003 INFO:teuthology.orchestra.run.smithi118.stdout:Enabling autotune for osd_memory_target 2024-09-17T08:27:44.504 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:44 smithi118 ceph-mon[29491]: mgrmap e13: y(active, since 6s) 2024-09-17T08:27:46.521 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-17T08:27:46.522 INFO:teuthology.orchestra.run.smithi118.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-17T08:27:46.522 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:27:46.522 INFO:teuthology.orchestra.run.smithi118.stdout: sudo /sbin/cephadm shell --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-17T08:27:46.522 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:27:46.522 INFO:teuthology.orchestra.run.smithi118.stdout:Or, if you are only running a single cluster on this host: 2024-09-17T08:27:46.522 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:27:46.522 INFO:teuthology.orchestra.run.smithi118.stdout: sudo /sbin/cephadm shell 2024-09-17T08:27:46.522 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:27:46.522 INFO:teuthology.orchestra.run.smithi118.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-17T08:27:46.522 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:27:46.522 INFO:teuthology.orchestra.run.smithi118.stdout: ceph telemetry on 2024-09-17T08:27:46.522 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:27:46.522 INFO:teuthology.orchestra.run.smithi118.stdout:For more information see: 2024-09-17T08:27:46.523 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:27:46.523 INFO:teuthology.orchestra.run.smithi118.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-17T08:27:46.523 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:27:46.523 INFO:teuthology.orchestra.run.smithi118.stdout:Bootstrap complete. 2024-09-17T08:27:46.554 INFO:tasks.cephadm:Fetching config... 2024-09-17T08:27:46.554 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-17T08:27:46.555 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-17T08:27:46.570 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-17T08:27:46.570 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-17T08:27:46.570 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-17T08:27:46.626 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-17T08:27:46.626 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-17T08:27:46.626 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-17T08:27:46.691 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-17T08:27:46.692 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-17T08:27:46.692 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-17T08:27:46.748 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-17T08:27:46.748 DEBUG:teuthology.orchestra.run.smithi118:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2ktbReshp3Nm7MfczDYGo0FBIJA02uqHgM1hk+SIFOFIgH1oEot+6ADddCsklol3GcvYffRJgFsCzqURiVgMAhChVzAekuYGb8/KPNmKTStmLpAz96Og+03EW00F7vN/5wbFAH9o2y1iYOVRcw72fldO4KJzlUnLM0A9CR7a91fwDKZaqQiIuyOQPbMgoe9pH4E7FepnYnT9ZSF556HADZaF8xIuT49uneDQoXiJO8ssvERyK55rezLLNfVqFSq/UZLUblzMegMzbFjAsSlc1aTiwyBWoVDHvZpjlw1wYHlKLhHD18x8YZyG778gjPj3h6TZNs7zrqA+B4+TXeWfg5kYbrfn9U1ANVTi1NF61Izb5sICW+86boOmwGxaFgrq1a5o4WGat+EEGKaZFPnmSS5tRkoIUEu6Gruv+GTHcno1J2tWmhWmCuB3GPaGUCn1xou+z1fNXODX6342mBrBtAQLmjbGvxFFfjgQ619S1riSChDz923uz1dXfb5Bhbzc= ceph-7862d192-74ce-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-17T08:27:46.827 INFO:teuthology.orchestra.run.smithi118.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2ktbReshp3Nm7MfczDYGo0FBIJA02uqHgM1hk+SIFOFIgH1oEot+6ADddCsklol3GcvYffRJgFsCzqURiVgMAhChVzAekuYGb8/KPNmKTStmLpAz96Og+03EW00F7vN/5wbFAH9o2y1iYOVRcw72fldO4KJzlUnLM0A9CR7a91fwDKZaqQiIuyOQPbMgoe9pH4E7FepnYnT9ZSF556HADZaF8xIuT49uneDQoXiJO8ssvERyK55rezLLNfVqFSq/UZLUblzMegMzbFjAsSlc1aTiwyBWoVDHvZpjlw1wYHlKLhHD18x8YZyG778gjPj3h6TZNs7zrqA+B4+TXeWfg5kYbrfn9U1ANVTi1NF61Izb5sICW+86boOmwGxaFgrq1a5o4WGat+EEGKaZFPnmSS5tRkoIUEu6Gruv+GTHcno1J2tWmhWmCuB3GPaGUCn1xou+z1fNXODX6342mBrBtAQLmjbGvxFFfjgQ619S1riSChDz923uz1dXfb5Bhbzc= ceph-7862d192-74ce-11ef-bceb-c7b262605968 2024-09-17T08:27:46.838 DEBUG:teuthology.orchestra.run.smithi170:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2ktbReshp3Nm7MfczDYGo0FBIJA02uqHgM1hk+SIFOFIgH1oEot+6ADddCsklol3GcvYffRJgFsCzqURiVgMAhChVzAekuYGb8/KPNmKTStmLpAz96Og+03EW00F7vN/5wbFAH9o2y1iYOVRcw72fldO4KJzlUnLM0A9CR7a91fwDKZaqQiIuyOQPbMgoe9pH4E7FepnYnT9ZSF556HADZaF8xIuT49uneDQoXiJO8ssvERyK55rezLLNfVqFSq/UZLUblzMegMzbFjAsSlc1aTiwyBWoVDHvZpjlw1wYHlKLhHD18x8YZyG778gjPj3h6TZNs7zrqA+B4+TXeWfg5kYbrfn9U1ANVTi1NF61Izb5sICW+86boOmwGxaFgrq1a5o4WGat+EEGKaZFPnmSS5tRkoIUEu6Gruv+GTHcno1J2tWmhWmCuB3GPaGUCn1xou+z1fNXODX6342mBrBtAQLmjbGvxFFfjgQ619S1riSChDz923uz1dXfb5Bhbzc= ceph-7862d192-74ce-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-17T08:27:46.904 INFO:teuthology.orchestra.run.smithi170.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2ktbReshp3Nm7MfczDYGo0FBIJA02uqHgM1hk+SIFOFIgH1oEot+6ADddCsklol3GcvYffRJgFsCzqURiVgMAhChVzAekuYGb8/KPNmKTStmLpAz96Og+03EW00F7vN/5wbFAH9o2y1iYOVRcw72fldO4KJzlUnLM0A9CR7a91fwDKZaqQiIuyOQPbMgoe9pH4E7FepnYnT9ZSF556HADZaF8xIuT49uneDQoXiJO8ssvERyK55rezLLNfVqFSq/UZLUblzMegMzbFjAsSlc1aTiwyBWoVDHvZpjlw1wYHlKLhHD18x8YZyG778gjPj3h6TZNs7zrqA+B4+TXeWfg5kYbrfn9U1ANVTi1NF61Izb5sICW+86boOmwGxaFgrq1a5o4WGat+EEGKaZFPnmSS5tRkoIUEu6Gruv+GTHcno1J2tWmhWmCuB3GPaGUCn1xou+z1fNXODX6342mBrBtAQLmjbGvxFFfjgQ619S1riSChDz923uz1dXfb5Bhbzc= ceph-7862d192-74ce-11ef-bceb-c7b262605968 2024-09-17T08:27:46.917 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-17T08:27:47.110 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:27:47.230 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:46 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/4282797206' entity='client.admin' 2024-09-17T08:27:50.144 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-17T08:27:50.144 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-17T08:27:50.468 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:27:50.893 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:50 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/1061105892' entity='client.admin' 2024-09-17T08:27:53.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:52 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:27:53.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:52 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:27:53.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:52 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi118", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:27:53.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:52 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:27:53.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:52 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:27:53.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:52 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:27:53.757 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi170 2024-09-17T08:27:53.758 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-17T08:27:53.758 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/etc/ceph/ceph.conf 2024-09-17T08:27:53.778 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-17T08:27:53.778 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:27:53.835 INFO:tasks.cephadm:Adding host smithi170 to orchestrator... 2024-09-17T08:27:53.836 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph orch host add smithi170 2024-09-17T08:27:54.029 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:27:54.266 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:54 smithi118 ceph-mon[29491]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:27:54.266 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:54 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:27:54.267 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:54 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:27:54.267 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:54 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:27:54.267 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:54 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:27:54.267 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:54 smithi118 ceph-mon[29491]: Updating smithi118:/etc/ceph/ceph.conf 2024-09-17T08:27:54.267 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:54 smithi118 ceph-mon[29491]: Updating smithi118:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:27:54.267 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:54 smithi118 ceph-mon[29491]: Updating smithi118:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:27:54.267 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:54 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:27:54.267 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:54 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:27:54.267 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:54 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:27:55.201 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:55 smithi118 ceph-mon[29491]: Updating smithi118:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:27:57.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:56 smithi118 ceph-mon[29491]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi170", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:27:58.270 INFO:teuthology.orchestra.run.smithi118.stdout:Added host 'smithi170' with addr '172.21.15.170' 2024-09-17T08:27:58.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:58 smithi118 ceph-mon[29491]: Deploying cephadm binary to smithi170 2024-09-17T08:27:58.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:58 smithi118 ceph-mon[29491]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:27:58.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:58 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:27:58.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:58 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:27:58.984 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-17T08:27:59.180 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:27:59.623 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:59 smithi118 ceph-mon[29491]: Added host smithi170 2024-09-17T08:27:59.623 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:59 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:27:59.624 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:27:59 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:00.608 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:00 smithi118 ceph-mon[29491]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:00.609 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:00 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:00.981 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:28:00.981 INFO:teuthology.orchestra.run.smithi118.stdout:[{"addr": "172.21.15.118", "hostname": "smithi118", "labels": [], "status": ""}, {"addr": "172.21.15.170", "hostname": "smithi170", "labels": [], "status": ""}] 2024-09-17T08:28:01.578 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-17T08:28:01.579 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-17T08:28:01.767 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:28:02.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:02 smithi118 ceph-mon[29491]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:28:02.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:02 smithi118 ceph-mon[29491]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:02.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:02 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:02.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:02 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:02.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:02 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:02.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:02 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:02.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:02 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi170", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:28:02.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:02 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:02.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:02 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:28:03.659 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:03 smithi118 ceph-mon[29491]: Updating smithi170:/etc/ceph/ceph.conf 2024-09-17T08:28:03.659 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:03 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:03.660 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:03 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:03.660 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:03 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:04.497 INFO:teuthology.orchestra.run.smithi118.stderr:adjusted tunables profile to default 2024-09-17T08:28:04.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:04 smithi118 ceph-mon[29491]: Updating smithi170:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:28:04.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:04 smithi118 ceph-mon[29491]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:28:04.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:04 smithi118 ceph-mon[29491]: Updating smithi170:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:28:04.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:04 smithi118 ceph-mon[29491]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:04.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:04 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/2997931594' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-17T08:28:05.109 INFO:tasks.cephadm:Adding mon.a on smithi118 2024-09-17T08:28:05.109 INFO:tasks.cephadm:Adding mon.c on smithi118 2024-09-17T08:28:05.109 INFO:tasks.cephadm:Adding mon.b on smithi170 2024-09-17T08:28:05.110 DEBUG:teuthology.orchestra.run.smithi170:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi118:172.21.15.118=a;smithi118:[v2:172.21.15.118:3301,v1:172.21.15.118:6790]=c;smithi170:172.21.15.170=b' 2024-09-17T08:28:05.306 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:28:05.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:05 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/2997931594' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-17T08:28:05.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:05 smithi118 ceph-mon[29491]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T08:28:06.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:06 smithi118 ceph-mon[29491]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:06.997 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled mon update... 2024-09-17T08:28:07.477 DEBUG:teuthology.orchestra.run.smithi118:mon.c> sudo journalctl -f -n 0 -u ceph-7862d192-74ce-11ef-bceb-c7b262605968@mon.c.service 2024-09-17T08:28:07.480 DEBUG:teuthology.orchestra.run.smithi170:mon.b> sudo journalctl -f -n 0 -u ceph-7862d192-74ce-11ef-bceb-c7b262605968@mon.b.service 2024-09-17T08:28:07.483 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-17T08:28:07.483 DEBUG:teuthology.orchestra.run.smithi170:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-17T08:28:08.253 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:28:08.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:07 smithi118 ceph-mon[29491]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi118:172.21.15.118=a;smithi118:[v2:172.21.15.118:3301,v1:172.21.15.118:6790]=c;smithi170:172.21.15.170=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:28:08.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:07 smithi118 ceph-mon[29491]: Saving service mon spec with placement smithi118:172.21.15.118=a;smithi118:[v2:172.21.15.118:3301,v1:172.21.15.118:6790]=c;smithi170:172.21.15.170=b;count:3 2024-09-17T08:28:08.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:07 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:08.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:07 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:28:08.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:07 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:08.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:07 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:28:08.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:07 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:08.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:07 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:28:08.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:07 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:08.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:07 smithi118 ceph-mon[29491]: Deploying daemon mon.b on smithi170 2024-09-17T08:28:08.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:07 smithi118 ceph-mon[29491]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:10.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:10 smithi118 ceph-mon[29491]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:11.487 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:28:11.487 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":1,"fsid":"7862d192-74ce-11ef-bceb-c7b262605968","modified":"2024-09-17T08:26:39.965247Z","created":"2024-09-17T08:26:39.965247Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:3300","nonce":0},{"type":"v1","addr":"172.21.15.118:6789","nonce":0}]},"addr":"172.21.15.118:6789/0","public_addr":"172.21.15.118:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-17T08:28:11.487 INFO:teuthology.orchestra.run.smithi170.stderr:dumped monmap epoch 1 2024-09-17T08:28:12.270 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 podman[33233]: 2024-09-17 08:28:12.182769437 +0000 UTC m=+0.122701365 container create c92809c44401538d290e476e6641c8490b95a3ee7bfe8079bc4ce2dbd5346c4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mon-b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731) 2024-09-17T08:28:12.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 podman[33233]: 2024-09-17 08:28:12.351640637 +0000 UTC m=+0.291572575 container init c92809c44401538d290e476e6641c8490b95a3ee7bfe8079bc4ce2dbd5346c4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mon-b, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-17T08:28:12.647 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 podman[33233]: 2024-09-17 08:28:12.356324483 +0000 UTC m=+0.296256411 container start c92809c44401538d290e476e6641c8490b95a3ee7bfe8079bc4ce2dbd5346c4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mon-b, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, GIT_CLEAN=True) 2024-09-17T08:28:12.647 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: set uid:gid to 167:167 (ceph:ceph) 2024-09-17T08:28:12.647 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable), process ceph-mon, pid 7 2024-09-17T08:28:12.647 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: pidfile_write: ignore empty --pid-file 2024-09-17T08:28:12.647 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: load: jerasure load: lrc 2024-09-17T08:28:12.648 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: RocksDB version: 7.9.2 2024-09-17T08:28:12.648 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Git sha 0 2024-09-17T08:28:12.648 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Compile date 2024-09-16 15:15:13 2024-09-17T08:28:12.648 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: DB SUMMARY 2024-09-17T08:28:12.648 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: DB Session ID: 9JDX0971Q6D1LFOVUUOH 2024-09-17T08:28:12.648 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: CURRENT file: CURRENT 2024-09-17T08:28:12.649 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: IDENTITY file: IDENTITY 2024-09-17T08:28:12.649 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-17T08:28:12.649 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-17T08:28:12.649 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 511 ; 2024-09-17T08:28:12.649 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.error_if_exists: 0 2024-09-17T08:28:12.650 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.create_if_missing: 0 2024-09-17T08:28:12.650 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.paranoid_checks: 1 2024-09-17T08:28:12.650 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-17T08:28:12.650 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-17T08:28:12.650 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-17T08:28:12.651 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.env: 0x55ca091cfbc0 2024-09-17T08:28:12.651 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.fs: PosixFileSystem 2024-09-17T08:28:12.651 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.info_log: 0x55ca0a8d0be0 2024-09-17T08:28:12.651 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.max_file_opening_threads: 16 2024-09-17T08:28:12.651 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.statistics: (nil) 2024-09-17T08:28:12.651 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.use_fsync: 0 2024-09-17T08:28:12.652 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.max_log_file_size: 0 2024-09-17T08:28:12.652 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-17T08:28:12.652 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-17T08:28:12.652 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.keep_log_file_num: 1000 2024-09-17T08:28:12.652 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.recycle_log_file_num: 0 2024-09-17T08:28:12.652 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.allow_fallocate: 1 2024-09-17T08:28:12.653 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.allow_mmap_reads: 0 2024-09-17T08:28:12.653 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.allow_mmap_writes: 0 2024-09-17T08:28:12.653 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.use_direct_reads: 0 2024-09-17T08:28:12.653 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-17T08:28:12.653 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.create_missing_column_families: 0 2024-09-17T08:28:12.653 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.db_log_dir: 2024-09-17T08:28:12.654 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.wal_dir: 2024-09-17T08:28:12.654 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-17T08:28:12.654 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-17T08:28:12.654 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-17T08:28:12.654 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-17T08:28:12.654 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-17T08:28:12.654 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-17T08:28:12.654 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.advise_random_on_open: 1 2024-09-17T08:28:12.654 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.db_write_buffer_size: 0 2024-09-17T08:28:12.654 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.write_buffer_manager: 0x55ca0a8de320 2024-09-17T08:28:12.655 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-17T08:28:12.655 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-17T08:28:12.655 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-17T08:28:12.655 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.rate_limiter: (nil) 2024-09-17T08:28:12.655 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-17T08:28:12.655 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.wal_recovery_mode: 2 2024-09-17T08:28:12.655 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.enable_thread_tracking: 0 2024-09-17T08:28:12.655 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.enable_pipelined_write: 0 2024-09-17T08:28:12.655 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.unordered_write: 0 2024-09-17T08:28:12.655 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-17T08:28:12.656 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-17T08:28:12.656 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-17T08:28:12.656 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-17T08:28:12.656 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.row_cache: None 2024-09-17T08:28:12.656 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.wal_filter: None 2024-09-17T08:28:12.656 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-17T08:28:12.656 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.allow_ingest_behind: 0 2024-09-17T08:28:12.656 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.two_write_queues: 0 2024-09-17T08:28:12.656 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.manual_wal_flush: 0 2024-09-17T08:28:12.657 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.wal_compression: 0 2024-09-17T08:28:12.657 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.atomic_flush: 0 2024-09-17T08:28:12.657 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-17T08:28:12.657 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-17T08:28:12.657 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-17T08:28:12.657 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.log_readahead_size: 0 2024-09-17T08:28:12.657 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-17T08:28:12.657 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.best_efforts_recovery: 0 2024-09-17T08:28:12.657 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-17T08:28:12.658 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-17T08:28:12.658 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.allow_data_in_errors: 0 2024-09-17T08:28:12.658 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.db_host_id: __hostname__ 2024-09-17T08:28:12.658 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-17T08:28:12.658 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.max_background_jobs: 2 2024-09-17T08:28:12.658 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.max_background_compactions: -1 2024-09-17T08:28:12.658 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.max_subcompactions: 1 2024-09-17T08:28:12.658 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-17T08:28:12.658 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-17T08:28:12.658 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-17T08:28:12.659 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.max_total_wal_size: 0 2024-09-17T08:28:12.659 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-17T08:28:12.659 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-17T08:28:12.659 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-17T08:28:12.659 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-17T08:28:12.659 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.max_open_files: -1 2024-09-17T08:28:12.659 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.bytes_per_sync: 0 2024-09-17T08:28:12.659 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-17T08:28:12.659 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-17T08:28:12.660 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.compaction_readahead_size: 0 2024-09-17T08:28:12.660 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.max_background_flushes: -1 2024-09-17T08:28:12.660 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Compression algorithms supported: 2024-09-17T08:28:12.660 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: kZSTD supported: 0 2024-09-17T08:28:12.660 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: kXpressCompression supported: 0 2024-09-17T08:28:12.660 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: kBZip2Compression supported: 0 2024-09-17T08:28:12.660 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-17T08:28:12.660 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: kLZ4Compression supported: 1 2024-09-17T08:28:12.660 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: kZlibCompression supported: 1 2024-09-17T08:28:12.660 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: kLZ4HCCompression supported: 1 2024-09-17T08:28:12.661 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: kSnappyCompression supported: 1 2024-09-17T08:28:12.661 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-17T08:28:12.661 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-17T08:28:12.661 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-17T08:28:12.661 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-17T08:28:12.661 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-17T08:28:12.661 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.merge_operator: 2024-09-17T08:28:12.661 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.compaction_filter: None 2024-09-17T08:28:12.661 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.compaction_filter_factory: None 2024-09-17T08:28:12.662 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.sst_partitioner_factory: None 2024-09-17T08:28:12.662 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-17T08:28:12.662 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-17T08:28:12.662 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ca0a8d07e0) 2024-09-17T08:28:12.662 INFO:journalctl@ceph.mon.b.smithi170.stdout: cache_index_and_filter_blocks: 1 2024-09-17T08:28:12.662 INFO:journalctl@ceph.mon.b.smithi170.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-17T08:28:12.662 INFO:journalctl@ceph.mon.b.smithi170.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-17T08:28:12.662 INFO:journalctl@ceph.mon.b.smithi170.stdout: pin_top_level_index_and_filter: 1 2024-09-17T08:28:12.662 INFO:journalctl@ceph.mon.b.smithi170.stdout: index_type: 0 2024-09-17T08:28:12.662 INFO:journalctl@ceph.mon.b.smithi170.stdout: data_block_index_type: 0 2024-09-17T08:28:12.663 INFO:journalctl@ceph.mon.b.smithi170.stdout: index_shortening: 1 2024-09-17T08:28:12.663 INFO:journalctl@ceph.mon.b.smithi170.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-17T08:28:12.663 INFO:journalctl@ceph.mon.b.smithi170.stdout: checksum: 4 2024-09-17T08:28:12.663 INFO:journalctl@ceph.mon.b.smithi170.stdout: no_block_cache: 0 2024-09-17T08:28:12.663 INFO:journalctl@ceph.mon.b.smithi170.stdout: block_cache: 0x55ca0a8c5350 2024-09-17T08:28:12.663 INFO:journalctl@ceph.mon.b.smithi170.stdout: block_cache_name: BinnedLRUCache 2024-09-17T08:28:12.663 INFO:journalctl@ceph.mon.b.smithi170.stdout: block_cache_options: 2024-09-17T08:28:12.663 INFO:journalctl@ceph.mon.b.smithi170.stdout: capacity : 536870912 2024-09-17T08:28:12.663 INFO:journalctl@ceph.mon.b.smithi170.stdout: num_shard_bits : 4 2024-09-17T08:28:12.664 INFO:journalctl@ceph.mon.b.smithi170.stdout: strict_capacity_limit : 0 2024-09-17T08:28:12.664 INFO:journalctl@ceph.mon.b.smithi170.stdout: high_pri_pool_ratio: 0.000 2024-09-17T08:28:12.664 INFO:journalctl@ceph.mon.b.smithi170.stdout: block_cache_compressed: (nil) 2024-09-17T08:28:12.664 INFO:journalctl@ceph.mon.b.smithi170.stdout: persistent_cache: (nil) 2024-09-17T08:28:12.664 INFO:journalctl@ceph.mon.b.smithi170.stdout: block_size: 4096 2024-09-17T08:28:12.664 INFO:journalctl@ceph.mon.b.smithi170.stdout: block_size_deviation: 10 2024-09-17T08:28:12.664 INFO:journalctl@ceph.mon.b.smithi170.stdout: block_restart_interval: 16 2024-09-17T08:28:12.664 INFO:journalctl@ceph.mon.b.smithi170.stdout: index_block_restart_interval: 1 2024-09-17T08:28:12.664 INFO:journalctl@ceph.mon.b.smithi170.stdout: metadata_block_size: 4096 2024-09-17T08:28:12.664 INFO:journalctl@ceph.mon.b.smithi170.stdout: partition_filters: 0 2024-09-17T08:28:12.665 INFO:journalctl@ceph.mon.b.smithi170.stdout: use_delta_encoding: 1 2024-09-17T08:28:12.665 INFO:journalctl@ceph.mon.b.smithi170.stdout: filter_policy: bloomfilter 2024-09-17T08:28:12.665 INFO:journalctl@ceph.mon.b.smithi170.stdout: whole_key_filtering: 1 2024-09-17T08:28:12.665 INFO:journalctl@ceph.mon.b.smithi170.stdout: verify_compression: 0 2024-09-17T08:28:12.665 INFO:journalctl@ceph.mon.b.smithi170.stdout: read_amp_bytes_per_bit: 0 2024-09-17T08:28:12.665 INFO:journalctl@ceph.mon.b.smithi170.stdout: format_version: 5 2024-09-17T08:28:12.665 INFO:journalctl@ceph.mon.b.smithi170.stdout: enable_index_compression: 1 2024-09-17T08:28:12.666 INFO:journalctl@ceph.mon.b.smithi170.stdout: block_align: 0 2024-09-17T08:28:12.666 INFO:journalctl@ceph.mon.b.smithi170.stdout: max_auto_readahead_size: 262144 2024-09-17T08:28:12.666 INFO:journalctl@ceph.mon.b.smithi170.stdout: prepopulate_block_cache: 0 2024-09-17T08:28:12.666 INFO:journalctl@ceph.mon.b.smithi170.stdout: initial_auto_readahead_size: 8192 2024-09-17T08:28:12.666 INFO:journalctl@ceph.mon.b.smithi170.stdout: num_file_reads_for_auto_readahead: 2 2024-09-17T08:28:12.668 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.write_buffer_size: 33554432 2024-09-17T08:28:12.669 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.max_write_buffer_number: 2 2024-09-17T08:28:12.669 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.compression: NoCompression 2024-09-17T08:28:12.669 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.bottommost_compression: Disabled 2024-09-17T08:28:12.669 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.prefix_extractor: nullptr 2024-09-17T08:28:12.669 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-17T08:28:12.669 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.num_levels: 7 2024-09-17T08:28:12.669 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-17T08:28:12.669 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-17T08:28:12.669 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-17T08:28:12.669 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-17T08:28:12.670 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-17T08:28:12.670 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-17T08:28:12.670 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-17T08:28:12.670 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-17T08:28:12.670 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-17T08:28:12.670 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-17T08:28:12.670 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-17T08:28:12.670 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-17T08:28:12.670 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-17T08:28:12.670 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.compression_opts.level: 32767 2024-09-17T08:28:12.670 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.compression_opts.strategy: 0 2024-09-17T08:28:12.670 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-17T08:28:12.670 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-17T08:28:12.670 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-17T08:28:12.671 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-17T08:28:12.671 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.compression_opts.enabled: false 2024-09-17T08:28:12.671 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-17T08:28:12.671 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-17T08:28:12.671 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-17T08:28:12.671 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-17T08:28:12.671 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.target_file_size_base: 67108864 2024-09-17T08:28:12.671 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-17T08:28:12.671 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-17T08:28:12.671 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-17T08:28:12.671 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-17T08:28:12.671 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-17T08:28:12.671 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-17T08:28:12.671 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-17T08:28:12.672 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-17T08:28:12.672 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-17T08:28:12.672 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-17T08:28:12.672 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-17T08:28:12.672 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-17T08:28:12.672 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-17T08:28:12.672 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-17T08:28:12.672 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.arena_block_size: 1048576 2024-09-17T08:28:12.672 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-17T08:28:12.672 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-17T08:28:12.672 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.disable_auto_compactions: 0 2024-09-17T08:28:12.673 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-17T08:28:12.673 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-17T08:28:12.673 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-17T08:28:12.673 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-17T08:28:12.673 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-17T08:28:12.673 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-17T08:28:12.673 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-17T08:28:12.673 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-17T08:28:12.673 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-17T08:28:12.673 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-17T08:28:12.673 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-17T08:28:12.673 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.inplace_update_support: 0 2024-09-17T08:28:12.673 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-17T08:28:12.673 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-17T08:28:12.674 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-17T08:28:12.674 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-17T08:28:12.674 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.bloom_locality: 0 2024-09-17T08:28:12.674 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.max_successive_merges: 0 2024-09-17T08:28:12.674 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-17T08:28:12.674 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.paranoid_file_checks: 0 2024-09-17T08:28:12.674 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.force_consistency_checks: 1 2024-09-17T08:28:12.674 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.report_bg_io_stats: 0 2024-09-17T08:28:12.674 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.ttl: 2592000 2024-09-17T08:28:12.674 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-17T08:28:12.674 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-17T08:28:12.674 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-17T08:28:12.674 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.enable_blob_files: false 2024-09-17T08:28:12.675 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.min_blob_size: 0 2024-09-17T08:28:12.675 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.blob_file_size: 268435456 2024-09-17T08:28:12.675 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-17T08:28:12.675 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-17T08:28:12.675 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-17T08:28:12.675 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-17T08:28:12.675 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-17T08:28:12.675 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.blob_file_starting_level: 0 2024-09-17T08:28:12.675 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-17T08:28:12.675 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-17T08:28:12.675 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-17T08:28:12.675 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 61fc7e03-2a92-4555-b6c4-23c0df11e059 2024-09-17T08:28:12.675 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726561692383648, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-17T08:28:12.675 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-17T08:28:12.676 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726561692384139, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726561692, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "61fc7e03-2a92-4555-b6c4-23c0df11e059", "db_session_id": "9JDX0971Q6D1LFOVUUOH", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-17T08:28:12.676 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726561692384231, "job": 1, "event": "recovery_finished"} 2024-09-17T08:28:12.676 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-17T08:28:12.676 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-17T08:28:12.676 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ca0a8f0e00 2024-09-17T08:28:12.676 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: DB pointer 0x55ca0a9e4000 2024-09-17T08:28:12.676 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-17T08:28:12.676 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-17T08:28:12.676 INFO:journalctl@ceph.mon.b.smithi170.stdout: ** DB Stats ** 2024-09-17T08:28:12.676 INFO:journalctl@ceph.mon.b.smithi170.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-17T08:28:12.676 INFO:journalctl@ceph.mon.b.smithi170.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-17T08:28:12.676 INFO:journalctl@ceph.mon.b.smithi170.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-17T08:28:12.676 INFO:journalctl@ceph.mon.b.smithi170.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-17T08:28:12.677 INFO:journalctl@ceph.mon.b.smithi170.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-17T08:28:12.677 INFO:journalctl@ceph.mon.b.smithi170.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-17T08:28:12.677 INFO:journalctl@ceph.mon.b.smithi170.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-17T08:28:12.677 INFO:journalctl@ceph.mon.b.smithi170.stdout: 2024-09-17T08:28:12.677 INFO:journalctl@ceph.mon.b.smithi170.stdout: ** Compaction Stats [default] ** 2024-09-17T08:28:12.677 INFO:journalctl@ceph.mon.b.smithi170.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-17T08:28:12.677 INFO:journalctl@ceph.mon.b.smithi170.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-17T08:28:12.677 INFO:journalctl@ceph.mon.b.smithi170.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T08:28:12.677 INFO:journalctl@ceph.mon.b.smithi170.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T08:28:12.677 INFO:journalctl@ceph.mon.b.smithi170.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T08:28:12.677 INFO:journalctl@ceph.mon.b.smithi170.stdout: 2024-09-17T08:28:12.677 INFO:journalctl@ceph.mon.b.smithi170.stdout: ** Compaction Stats [default] ** 2024-09-17T08:28:12.677 INFO:journalctl@ceph.mon.b.smithi170.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-17T08:28:12.677 INFO:journalctl@ceph.mon.b.smithi170.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-17T08:28:12.678 INFO:journalctl@ceph.mon.b.smithi170.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T08:28:12.678 INFO:journalctl@ceph.mon.b.smithi170.stdout: 2024-09-17T08:28:12.678 INFO:journalctl@ceph.mon.b.smithi170.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-17T08:28:12.678 INFO:journalctl@ceph.mon.b.smithi170.stdout: 2024-09-17T08:28:12.678 INFO:journalctl@ceph.mon.b.smithi170.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-17T08:28:12.678 INFO:journalctl@ceph.mon.b.smithi170.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-17T08:28:12.678 INFO:journalctl@ceph.mon.b.smithi170.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-17T08:28:12.678 INFO:journalctl@ceph.mon.b.smithi170.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-17T08:28:12.678 INFO:journalctl@ceph.mon.b.smithi170.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-17T08:28:12.678 INFO:journalctl@ceph.mon.b.smithi170.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-17T08:28:12.678 INFO:journalctl@ceph.mon.b.smithi170.stdout: Cumulative compaction: 0.00 GB write, 0.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-17T08:28:12.678 INFO:journalctl@ceph.mon.b.smithi170.stdout: Interval compaction: 0.00 GB write, 0.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-17T08:28:12.678 INFO:journalctl@ceph.mon.b.smithi170.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-17T08:28:12.678 INFO:journalctl@ceph.mon.b.smithi170.stdout: Block cache BinnedLRUCache@0x55ca0a8c5350#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-17T08:28:12.679 INFO:journalctl@ceph.mon.b.smithi170.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-17T08:28:12.679 INFO:journalctl@ceph.mon.b.smithi170.stdout: 2024-09-17T08:28:12.679 INFO:journalctl@ceph.mon.b.smithi170.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-17T08:28:12.679 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-17T08:28:12.679 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: using public_addr v2:172.21.15.170:0/0 -> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] 2024-09-17T08:28:12.679 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: starting mon.b rank -1 at public addrs [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] at bind addrs [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 7862d192-74ce-11ef-bceb-c7b262605968 2024-09-17T08:28:12.679 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: mon.b@-1(???) e0 preinit fsid 7862d192-74ce-11ef-bceb-c7b262605968 2024-09-17T08:28:12.679 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: mon.b@-1(synchronizing).mds e1 new map 2024-09-17T08:28:12.679 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-17T08:28:12.679 INFO:journalctl@ceph.mon.b.smithi170.stdout: e1 2024-09-17T08:28:12.679 INFO:journalctl@ceph.mon.b.smithi170.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T08:28:12.679 INFO:journalctl@ceph.mon.b.smithi170.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T08:28:12.679 INFO:journalctl@ceph.mon.b.smithi170.stdout: legacy client fscid: -1 2024-09-17T08:28:12.679 INFO:journalctl@ceph.mon.b.smithi170.stdout: 2024-09-17T08:28:12.680 INFO:journalctl@ceph.mon.b.smithi170.stdout: No filesystems configured 2024-09-17T08:28:12.680 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-17T08:28:12.680 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-17T08:28:12.680 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-17T08:28:12.680 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-17T08:28:12.680 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-17T08:28:12.680 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-17T08:28:12.680 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-17T08:28:12.680 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T08:28:12.680 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T08:28:12.680 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T08:28:12.680 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: Updating smithi170:/etc/ceph/ceph.conf 2024-09-17T08:28:12.680 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:12.680 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:12.681 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:12.681 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: Updating smithi170:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:28:12.681 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:28:12.681 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: Updating smithi170:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:28:12.681 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:12.681 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/2997931594' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-17T08:28:12.681 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/2997931594' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-17T08:28:12.681 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T08:28:12.681 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:12.681 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi118:172.21.15.118=a;smithi118:[v2:172.21.15.118:3301,v1:172.21.15.118:6790]=c;smithi170:172.21.15.170=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:28:12.681 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: Saving service mon spec with placement smithi118:172.21.15.118=a;smithi118:[v2:172.21.15.118:3301,v1:172.21.15.118:6790]=c;smithi170:172.21.15.170=b;count:3 2024-09-17T08:28:12.681 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:12.681 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:28:12.682 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:12.682 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:28:12.682 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:12.682 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:28:12.682 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:12.682 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: Deploying daemon mon.b on smithi170 2024-09-17T08:28:12.682 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:12.682 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:12.682 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:12.682 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: from='client.? 172.21.15.170:0/1540385400' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T08:28:12.682 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 ceph-mon[33271]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-17T08:28:12.682 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 bash[33233]: c92809c44401538d290e476e6641c8490b95a3ee7bfe8079bc4ce2dbd5346c4e 2024-09-17T08:28:12.682 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:12 smithi170 systemd[1]: Started Ceph mon.b for 7862d192-74ce-11ef-bceb-c7b262605968. 2024-09-17T08:28:12.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:12 smithi118 ceph-mon[29491]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:12.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:12 smithi118 ceph-mon[29491]: from='client.? 172.21.15.170:0/1540385400' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T08:28:13.037 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-17T08:28:13.038 DEBUG:teuthology.orchestra.run.smithi170:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-17T08:28:13.246 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:28:17.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:17 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:28:17.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:17 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:28:17.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:17 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:28:17.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:17 smithi118 ceph-mon[29491]: mon.a calling monitor election 2024-09-17T08:28:17.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:17 smithi118 ceph-mon[29491]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:17.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:17 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:28:17.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:17 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:28:17.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:17 smithi118 ceph-mon[29491]: mon.b calling monitor election 2024-09-17T08:28:17.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:17 smithi118 ceph-mon[29491]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:17.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:17 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:28:17.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:17 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:28:17.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:17 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:28:17.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:17 smithi118 ceph-mon[29491]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-17T08:28:17.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:17 smithi118 ceph-mon[29491]: monmap e2: 2 mons at {a=[v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0],b=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T08:28:17.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:17 smithi118 ceph-mon[29491]: fsmap 2024-09-17T08:28:17.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:17 smithi118 ceph-mon[29491]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T08:28:17.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:17 smithi118 ceph-mon[29491]: mgrmap e13: y(active, since 40s) 2024-09-17T08:28:17.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:17 smithi118 ceph-mon[29491]: overall HEALTH_OK 2024-09-17T08:28:17.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:17 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:17.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:17 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:17.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:17 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:17.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:17 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:28:17.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:17 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:17.869 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:17 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:28:17.870 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:17 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:28:17.870 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:17 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:28:17.870 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:17 smithi170 ceph-mon[33271]: mon.a calling monitor election 2024-09-17T08:28:17.870 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:17 smithi170 ceph-mon[33271]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:17.870 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:17 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:28:17.870 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:17 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:28:17.870 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:17 smithi170 ceph-mon[33271]: mon.b calling monitor election 2024-09-17T08:28:17.870 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:17 smithi170 ceph-mon[33271]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:17.870 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:17 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:28:17.870 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:17 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:28:17.871 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:17 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:28:17.871 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:17 smithi170 ceph-mon[33271]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-17T08:28:17.871 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:17 smithi170 ceph-mon[33271]: monmap e2: 2 mons at {a=[v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0],b=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T08:28:17.871 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:17 smithi170 ceph-mon[33271]: fsmap 2024-09-17T08:28:17.871 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:17 smithi170 ceph-mon[33271]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T08:28:17.871 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:17 smithi170 ceph-mon[33271]: mgrmap e13: y(active, since 40s) 2024-09-17T08:28:17.871 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:17 smithi170 ceph-mon[33271]: overall HEALTH_OK 2024-09-17T08:28:17.871 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:17 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:17.871 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:17 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:17.871 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:17 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:17.872 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:17 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:28:17.872 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:17 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:17.968 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:28:17.969 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":2,"fsid":"7862d192-74ce-11ef-bceb-c7b262605968","modified":"2024-09-17T08:28:12.408181Z","created":"2024-09-17T08:26:39.965247Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:3300","nonce":0},{"type":"v1","addr":"172.21.15.118:6789","nonce":0}]},"addr":"172.21.15.118:6789/0","public_addr":"172.21.15.118:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:3300","nonce":0},{"type":"v1","addr":"172.21.15.170:6789","nonce":0}]},"addr":"172.21.15.170:6789/0","public_addr":"172.21.15.170:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-17T08:28:17.969 INFO:teuthology.orchestra.run.smithi170.stderr:dumped monmap epoch 2 2024-09-17T08:28:18.571 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:18 smithi170 ceph-mon[33271]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:18.571 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:18 smithi170 ceph-mon[33271]: Deploying daemon mon.c on smithi118 2024-09-17T08:28:18.571 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:18 smithi170 ceph-mon[33271]: from='client.? 172.21.15.170:0/4211370730' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T08:28:18.571 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:18 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:28:18.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:18 smithi118 ceph-mon[29491]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:18.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:18 smithi118 ceph-mon[29491]: Deploying daemon mon.c on smithi118 2024-09-17T08:28:18.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:18 smithi118 ceph-mon[29491]: from='client.? 172.21.15.170:0/4211370730' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T08:28:18.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:18 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:28:19.593 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-17T08:28:19.595 DEBUG:teuthology.orchestra.run.smithi170:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-17T08:28:19.792 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:28:20.700 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:20 smithi118 ceph-mon[29491]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:20.700 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:20 smithi118 systemd[1]: Starting Ceph mon.c for 7862d192-74ce-11ef-bceb-c7b262605968... 2024-09-17T08:28:20.751 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:20 smithi170 ceph-mon[33271]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:20.971 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:20 smithi118 podman[41380]: 2024-09-17 08:28:20.602709495 +0000 UTC m=+0.015437941 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T08:28:20.971 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:20 smithi118 podman[41380]: 2024-09-17 08:28:20.710983332 +0000 UTC m=+0.123711763 container create 4cd40efe8dc5fe4f1d8a866d0890747eb0a3776890e93ca540d9ba1a40f2f297 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mon-c, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, RELEASE=reef-19e2a2b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-17T08:28:20.971 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:20 smithi118 podman[41380]: 2024-09-17 08:28:20.971375718 +0000 UTC m=+0.384104164 container init 4cd40efe8dc5fe4f1d8a866d0890747eb0a3776890e93ca540d9ba1a40f2f297 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mon-c, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-17T08:28:21.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:20 smithi118 podman[41380]: 2024-09-17 08:28:20.976067434 +0000 UTC m=+0.388795865 container start 4cd40efe8dc5fe4f1d8a866d0890747eb0a3776890e93ca540d9ba1a40f2f297 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mon-c, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=reef-19e2a2b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-17T08:28:21.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:20 smithi118 ceph-mon[41418]: set uid:gid to 167:167 (ceph:ceph) 2024-09-17T08:28:21.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:20 smithi118 ceph-mon[41418]: ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable), process ceph-mon, pid 7 2024-09-17T08:28:21.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:20 smithi118 ceph-mon[41418]: pidfile_write: ignore empty --pid-file 2024-09-17T08:28:21.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: load: jerasure load: lrc 2024-09-17T08:28:21.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: RocksDB version: 7.9.2 2024-09-17T08:28:21.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Git sha 0 2024-09-17T08:28:21.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Compile date 2024-09-16 15:15:13 2024-09-17T08:28:21.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: DB SUMMARY 2024-09-17T08:28:21.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: DB Session ID: HOSV2X1ZM2AO4QRPY21F 2024-09-17T08:28:21.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: CURRENT file: CURRENT 2024-09-17T08:28:21.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: IDENTITY file: IDENTITY 2024-09-17T08:28:21.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-17T08:28:21.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-17T08:28:21.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 636 ; 2024-09-17T08:28:21.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.error_if_exists: 0 2024-09-17T08:28:21.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.create_if_missing: 0 2024-09-17T08:28:21.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.paranoid_checks: 1 2024-09-17T08:28:21.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-17T08:28:21.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-17T08:28:21.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-17T08:28:21.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.env: 0x560f52be6bc0 2024-09-17T08:28:21.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.fs: PosixFileSystem 2024-09-17T08:28:21.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.info_log: 0x560f551eabe0 2024-09-17T08:28:21.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.max_file_opening_threads: 16 2024-09-17T08:28:21.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.statistics: (nil) 2024-09-17T08:28:21.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.use_fsync: 0 2024-09-17T08:28:21.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.max_log_file_size: 0 2024-09-17T08:28:21.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-17T08:28:21.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-17T08:28:21.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.keep_log_file_num: 1000 2024-09-17T08:28:21.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.recycle_log_file_num: 0 2024-09-17T08:28:21.280 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.allow_fallocate: 1 2024-09-17T08:28:21.281 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.allow_mmap_reads: 0 2024-09-17T08:28:21.281 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.allow_mmap_writes: 0 2024-09-17T08:28:21.281 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.use_direct_reads: 0 2024-09-17T08:28:21.281 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-17T08:28:21.281 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.create_missing_column_families: 0 2024-09-17T08:28:21.281 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.db_log_dir: 2024-09-17T08:28:21.282 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.wal_dir: 2024-09-17T08:28:21.282 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-17T08:28:21.282 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-17T08:28:21.282 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-17T08:28:21.282 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-17T08:28:21.282 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-17T08:28:21.282 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-17T08:28:21.282 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.advise_random_on_open: 1 2024-09-17T08:28:21.282 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.db_write_buffer_size: 0 2024-09-17T08:28:21.282 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.write_buffer_manager: 0x560f551f8320 2024-09-17T08:28:21.282 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-17T08:28:21.282 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-17T08:28:21.282 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-17T08:28:21.282 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.rate_limiter: (nil) 2024-09-17T08:28:21.283 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-17T08:28:21.283 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.wal_recovery_mode: 2 2024-09-17T08:28:21.283 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.enable_thread_tracking: 0 2024-09-17T08:28:21.283 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.enable_pipelined_write: 0 2024-09-17T08:28:21.283 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.unordered_write: 0 2024-09-17T08:28:21.283 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-17T08:28:21.283 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-17T08:28:21.283 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-17T08:28:21.283 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-17T08:28:21.283 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.row_cache: None 2024-09-17T08:28:21.284 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.wal_filter: None 2024-09-17T08:28:21.284 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-17T08:28:21.284 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.allow_ingest_behind: 0 2024-09-17T08:28:21.284 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.two_write_queues: 0 2024-09-17T08:28:21.284 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.manual_wal_flush: 0 2024-09-17T08:28:21.284 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.wal_compression: 0 2024-09-17T08:28:21.284 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.atomic_flush: 0 2024-09-17T08:28:21.284 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-17T08:28:21.284 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-17T08:28:21.284 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-17T08:28:21.284 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.log_readahead_size: 0 2024-09-17T08:28:21.284 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-17T08:28:21.284 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.best_efforts_recovery: 0 2024-09-17T08:28:21.285 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-17T08:28:21.285 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-17T08:28:21.285 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.allow_data_in_errors: 0 2024-09-17T08:28:21.285 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.db_host_id: __hostname__ 2024-09-17T08:28:21.285 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-17T08:28:21.285 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.max_background_jobs: 2 2024-09-17T08:28:21.285 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.max_background_compactions: -1 2024-09-17T08:28:21.285 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.max_subcompactions: 1 2024-09-17T08:28:21.285 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-17T08:28:21.285 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-17T08:28:21.285 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-17T08:28:21.285 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.max_total_wal_size: 0 2024-09-17T08:28:21.285 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-17T08:28:21.285 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-17T08:28:21.286 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-17T08:28:21.286 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-17T08:28:21.286 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.max_open_files: -1 2024-09-17T08:28:21.286 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.bytes_per_sync: 0 2024-09-17T08:28:21.286 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-17T08:28:21.286 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-17T08:28:21.286 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.compaction_readahead_size: 0 2024-09-17T08:28:21.286 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.max_background_flushes: -1 2024-09-17T08:28:21.286 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Compression algorithms supported: 2024-09-17T08:28:21.286 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: kZSTD supported: 0 2024-09-17T08:28:21.286 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: kXpressCompression supported: 0 2024-09-17T08:28:21.286 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: kBZip2Compression supported: 0 2024-09-17T08:28:21.286 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-17T08:28:21.286 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: kLZ4Compression supported: 1 2024-09-17T08:28:21.287 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: kZlibCompression supported: 1 2024-09-17T08:28:21.287 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: kLZ4HCCompression supported: 1 2024-09-17T08:28:21.287 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: kSnappyCompression supported: 1 2024-09-17T08:28:21.287 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-17T08:28:21.287 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-17T08:28:21.287 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-17T08:28:21.287 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-17T08:28:21.287 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-17T08:28:21.287 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.merge_operator: 2024-09-17T08:28:21.287 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.compaction_filter: None 2024-09-17T08:28:21.287 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.compaction_filter_factory: None 2024-09-17T08:28:21.288 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.sst_partitioner_factory: None 2024-09-17T08:28:21.288 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-17T08:28:21.288 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-17T08:28:21.288 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560f551ea7e0) 2024-09-17T08:28:21.288 INFO:journalctl@ceph.mon.c.smithi118.stdout: cache_index_and_filter_blocks: 1 2024-09-17T08:28:21.288 INFO:journalctl@ceph.mon.c.smithi118.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-17T08:28:21.288 INFO:journalctl@ceph.mon.c.smithi118.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-17T08:28:21.288 INFO:journalctl@ceph.mon.c.smithi118.stdout: pin_top_level_index_and_filter: 1 2024-09-17T08:28:21.288 INFO:journalctl@ceph.mon.c.smithi118.stdout: index_type: 0 2024-09-17T08:28:21.288 INFO:journalctl@ceph.mon.c.smithi118.stdout: data_block_index_type: 0 2024-09-17T08:28:21.288 INFO:journalctl@ceph.mon.c.smithi118.stdout: index_shortening: 1 2024-09-17T08:28:21.288 INFO:journalctl@ceph.mon.c.smithi118.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-17T08:28:21.288 INFO:journalctl@ceph.mon.c.smithi118.stdout: checksum: 4 2024-09-17T08:28:21.289 INFO:journalctl@ceph.mon.c.smithi118.stdout: no_block_cache: 0 2024-09-17T08:28:21.289 INFO:journalctl@ceph.mon.c.smithi118.stdout: block_cache: 0x560f551df350 2024-09-17T08:28:21.289 INFO:journalctl@ceph.mon.c.smithi118.stdout: block_cache_name: BinnedLRUCache 2024-09-17T08:28:21.289 INFO:journalctl@ceph.mon.c.smithi118.stdout: block_cache_options: 2024-09-17T08:28:21.289 INFO:journalctl@ceph.mon.c.smithi118.stdout: capacity : 536870912 2024-09-17T08:28:21.289 INFO:journalctl@ceph.mon.c.smithi118.stdout: num_shard_bits : 4 2024-09-17T08:28:21.289 INFO:journalctl@ceph.mon.c.smithi118.stdout: strict_capacity_limit : 0 2024-09-17T08:28:21.289 INFO:journalctl@ceph.mon.c.smithi118.stdout: high_pri_pool_ratio: 0.000 2024-09-17T08:28:21.289 INFO:journalctl@ceph.mon.c.smithi118.stdout: block_cache_compressed: (nil) 2024-09-17T08:28:21.289 INFO:journalctl@ceph.mon.c.smithi118.stdout: persistent_cache: (nil) 2024-09-17T08:28:21.289 INFO:journalctl@ceph.mon.c.smithi118.stdout: block_size: 4096 2024-09-17T08:28:21.289 INFO:journalctl@ceph.mon.c.smithi118.stdout: block_size_deviation: 10 2024-09-17T08:28:21.289 INFO:journalctl@ceph.mon.c.smithi118.stdout: block_restart_interval: 16 2024-09-17T08:28:21.289 INFO:journalctl@ceph.mon.c.smithi118.stdout: index_block_restart_interval: 1 2024-09-17T08:28:21.290 INFO:journalctl@ceph.mon.c.smithi118.stdout: metadata_block_size: 4096 2024-09-17T08:28:21.290 INFO:journalctl@ceph.mon.c.smithi118.stdout: partition_filters: 0 2024-09-17T08:28:21.290 INFO:journalctl@ceph.mon.c.smithi118.stdout: use_delta_encoding: 1 2024-09-17T08:28:21.290 INFO:journalctl@ceph.mon.c.smithi118.stdout: filter_policy: bloomfilter 2024-09-17T08:28:21.290 INFO:journalctl@ceph.mon.c.smithi118.stdout: whole_key_filtering: 1 2024-09-17T08:28:21.290 INFO:journalctl@ceph.mon.c.smithi118.stdout: verify_compression: 0 2024-09-17T08:28:21.290 INFO:journalctl@ceph.mon.c.smithi118.stdout: read_amp_bytes_per_bit: 0 2024-09-17T08:28:21.290 INFO:journalctl@ceph.mon.c.smithi118.stdout: format_version: 5 2024-09-17T08:28:21.290 INFO:journalctl@ceph.mon.c.smithi118.stdout: enable_index_compression: 1 2024-09-17T08:28:21.290 INFO:journalctl@ceph.mon.c.smithi118.stdout: block_align: 0 2024-09-17T08:28:21.290 INFO:journalctl@ceph.mon.c.smithi118.stdout: max_auto_readahead_size: 262144 2024-09-17T08:28:21.290 INFO:journalctl@ceph.mon.c.smithi118.stdout: prepopulate_block_cache: 0 2024-09-17T08:28:21.290 INFO:journalctl@ceph.mon.c.smithi118.stdout: initial_auto_readahead_size: 8192 2024-09-17T08:28:21.290 INFO:journalctl@ceph.mon.c.smithi118.stdout: num_file_reads_for_auto_readahead: 2 2024-09-17T08:28:21.290 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.write_buffer_size: 33554432 2024-09-17T08:28:21.291 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.max_write_buffer_number: 2 2024-09-17T08:28:21.291 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.compression: NoCompression 2024-09-17T08:28:21.291 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.bottommost_compression: Disabled 2024-09-17T08:28:21.291 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.prefix_extractor: nullptr 2024-09-17T08:28:21.291 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-17T08:28:21.291 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.num_levels: 7 2024-09-17T08:28:21.291 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-17T08:28:21.291 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-17T08:28:21.291 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-17T08:28:21.291 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-17T08:28:21.291 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-17T08:28:21.291 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-17T08:28:21.291 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-17T08:28:21.291 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-17T08:28:21.292 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-17T08:28:21.292 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-17T08:28:21.292 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-17T08:28:21.292 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-17T08:28:21.292 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-17T08:28:21.292 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.compression_opts.level: 32767 2024-09-17T08:28:21.292 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.compression_opts.strategy: 0 2024-09-17T08:28:21.292 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-17T08:28:21.292 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-17T08:28:21.292 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-17T08:28:21.292 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-17T08:28:21.292 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.compression_opts.enabled: false 2024-09-17T08:28:21.292 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-17T08:28:21.292 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-17T08:28:21.293 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-17T08:28:21.293 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-17T08:28:21.293 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.target_file_size_base: 67108864 2024-09-17T08:28:21.293 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-17T08:28:21.293 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-17T08:28:21.293 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-17T08:28:21.293 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-17T08:28:21.293 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-17T08:28:21.293 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-17T08:28:21.293 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-17T08:28:21.293 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-17T08:28:21.293 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-17T08:28:21.294 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-17T08:28:21.294 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-17T08:28:21.294 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-17T08:28:21.294 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-17T08:28:21.294 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-17T08:28:21.294 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.arena_block_size: 1048576 2024-09-17T08:28:21.294 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-17T08:28:21.294 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-17T08:28:21.294 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.disable_auto_compactions: 0 2024-09-17T08:28:21.294 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-17T08:28:21.294 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-17T08:28:21.294 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-17T08:28:21.294 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-17T08:28:21.295 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-17T08:28:21.295 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-17T08:28:21.295 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-17T08:28:21.295 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-17T08:28:21.295 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-17T08:28:21.295 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-17T08:28:21.295 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-17T08:28:21.295 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.inplace_update_support: 0 2024-09-17T08:28:21.295 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-17T08:28:21.295 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-17T08:28:21.295 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-17T08:28:21.295 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-17T08:28:21.295 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.bloom_locality: 0 2024-09-17T08:28:21.295 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.max_successive_merges: 0 2024-09-17T08:28:21.296 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-17T08:28:21.296 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.paranoid_file_checks: 0 2024-09-17T08:28:21.296 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.force_consistency_checks: 1 2024-09-17T08:28:21.296 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.report_bg_io_stats: 0 2024-09-17T08:28:21.296 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.ttl: 2592000 2024-09-17T08:28:21.296 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-17T08:28:21.296 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-17T08:28:21.296 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-17T08:28:21.296 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.enable_blob_files: false 2024-09-17T08:28:21.296 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.min_blob_size: 0 2024-09-17T08:28:21.296 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.blob_file_size: 268435456 2024-09-17T08:28:21.296 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-17T08:28:21.296 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-17T08:28:21.297 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-17T08:28:21.297 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-17T08:28:21.297 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-17T08:28:21.297 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.blob_file_starting_level: 0 2024-09-17T08:28:21.297 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-17T08:28:21.297 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-17T08:28:21.297 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-17T08:28:21.297 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: eaf3d000-4de9-4f19-b09a-d3618ed9a185 2024-09-17T08:28:21.297 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726561701002363, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-17T08:28:21.297 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-17T08:28:21.297 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726561701002756, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726561701, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "eaf3d000-4de9-4f19-b09a-d3618ed9a185", "db_session_id": "HOSV2X1ZM2AO4QRPY21F", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-17T08:28:21.297 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726561701002852, "job": 1, "event": "recovery_finished"} 2024-09-17T08:28:21.298 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-17T08:28:21.298 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-17T08:28:21.298 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560f5520ae00 2024-09-17T08:28:21.298 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: DB pointer 0x560f552fe000 2024-09-17T08:28:21.298 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-17T08:28:21.298 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-17T08:28:21.298 INFO:journalctl@ceph.mon.c.smithi118.stdout: ** DB Stats ** 2024-09-17T08:28:21.298 INFO:journalctl@ceph.mon.c.smithi118.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-17T08:28:21.298 INFO:journalctl@ceph.mon.c.smithi118.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-17T08:28:21.298 INFO:journalctl@ceph.mon.c.smithi118.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-17T08:28:21.298 INFO:journalctl@ceph.mon.c.smithi118.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-17T08:28:21.298 INFO:journalctl@ceph.mon.c.smithi118.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-17T08:28:21.298 INFO:journalctl@ceph.mon.c.smithi118.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-17T08:28:21.298 INFO:journalctl@ceph.mon.c.smithi118.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-17T08:28:21.299 INFO:journalctl@ceph.mon.c.smithi118.stdout: 2024-09-17T08:28:21.299 INFO:journalctl@ceph.mon.c.smithi118.stdout: ** Compaction Stats [default] ** 2024-09-17T08:28:21.299 INFO:journalctl@ceph.mon.c.smithi118.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-17T08:28:21.299 INFO:journalctl@ceph.mon.c.smithi118.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-17T08:28:21.299 INFO:journalctl@ceph.mon.c.smithi118.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T08:28:21.299 INFO:journalctl@ceph.mon.c.smithi118.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T08:28:21.299 INFO:journalctl@ceph.mon.c.smithi118.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T08:28:21.299 INFO:journalctl@ceph.mon.c.smithi118.stdout: 2024-09-17T08:28:21.299 INFO:journalctl@ceph.mon.c.smithi118.stdout: ** Compaction Stats [default] ** 2024-09-17T08:28:21.299 INFO:journalctl@ceph.mon.c.smithi118.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-17T08:28:21.299 INFO:journalctl@ceph.mon.c.smithi118.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-17T08:28:21.299 INFO:journalctl@ceph.mon.c.smithi118.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T08:28:21.299 INFO:journalctl@ceph.mon.c.smithi118.stdout: 2024-09-17T08:28:21.299 INFO:journalctl@ceph.mon.c.smithi118.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-17T08:28:21.300 INFO:journalctl@ceph.mon.c.smithi118.stdout: 2024-09-17T08:28:21.300 INFO:journalctl@ceph.mon.c.smithi118.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-17T08:28:21.300 INFO:journalctl@ceph.mon.c.smithi118.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-17T08:28:21.300 INFO:journalctl@ceph.mon.c.smithi118.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-17T08:28:21.300 INFO:journalctl@ceph.mon.c.smithi118.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-17T08:28:21.300 INFO:journalctl@ceph.mon.c.smithi118.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-17T08:28:21.300 INFO:journalctl@ceph.mon.c.smithi118.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-17T08:28:21.300 INFO:journalctl@ceph.mon.c.smithi118.stdout: Cumulative compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-17T08:28:21.300 INFO:journalctl@ceph.mon.c.smithi118.stdout: Interval compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-17T08:28:21.300 INFO:journalctl@ceph.mon.c.smithi118.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-17T08:28:21.300 INFO:journalctl@ceph.mon.c.smithi118.stdout: Block cache BinnedLRUCache@0x560f551df350#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-09-17T08:28:21.300 INFO:journalctl@ceph.mon.c.smithi118.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-17T08:28:21.300 INFO:journalctl@ceph.mon.c.smithi118.stdout: 2024-09-17T08:28:21.301 INFO:journalctl@ceph.mon.c.smithi118.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-17T08:28:21.301 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-17T08:28:21.301 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: using public_addrv [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] 2024-09-17T08:28:21.301 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: starting mon.c rank -1 at public addrs [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] at bind addrs [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid 7862d192-74ce-11ef-bceb-c7b262605968 2024-09-17T08:28:21.301 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: mon.c@-1(???) e0 preinit fsid 7862d192-74ce-11ef-bceb-c7b262605968 2024-09-17T08:28:21.301 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: mon.c@-1(synchronizing).mds e1 new map 2024-09-17T08:28:21.301 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-17T08:28:21.301 INFO:journalctl@ceph.mon.c.smithi118.stdout: e1 2024-09-17T08:28:21.301 INFO:journalctl@ceph.mon.c.smithi118.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T08:28:21.301 INFO:journalctl@ceph.mon.c.smithi118.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T08:28:21.301 INFO:journalctl@ceph.mon.c.smithi118.stdout: legacy client fscid: -1 2024-09-17T08:28:21.301 INFO:journalctl@ceph.mon.c.smithi118.stdout: 2024-09-17T08:28:21.301 INFO:journalctl@ceph.mon.c.smithi118.stdout: No filesystems configured 2024-09-17T08:28:21.301 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-17T08:28:21.302 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-17T08:28:21.302 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-17T08:28:21.302 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-17T08:28:21.302 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-17T08:28:21.302 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-17T08:28:21.302 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-17T08:28:21.302 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T08:28:21.302 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T08:28:21.302 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T08:28:21.302 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: Updating smithi170:/etc/ceph/ceph.conf 2024-09-17T08:28:21.302 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:21.302 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:21.302 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:21.302 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: Updating smithi170:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:28:21.303 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:28:21.303 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: Updating smithi170:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:28:21.303 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:21.303 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/2997931594' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-17T08:28:21.303 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/2997931594' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-17T08:28:21.303 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T08:28:21.303 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:21.303 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi118:172.21.15.118=a;smithi118:[v2:172.21.15.118:3301,v1:172.21.15.118:6790]=c;smithi170:172.21.15.170=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:28:21.303 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: Saving service mon spec with placement smithi118:172.21.15.118=a;smithi118:[v2:172.21.15.118:3301,v1:172.21.15.118:6790]=c;smithi170:172.21.15.170=b;count:3 2024-09-17T08:28:21.303 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:21.303 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:28:21.303 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:21.303 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:28:21.303 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:21.304 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:28:21.304 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:21.304 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: Deploying daemon mon.b on smithi170 2024-09-17T08:28:21.304 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:21.304 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:21.304 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:21.304 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: from='client.? 172.21.15.170:0/1540385400' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T08:28:21.304 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:28:21.304 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:28:21.304 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:28:21.304 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: mon.a calling monitor election 2024-09-17T08:28:21.304 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:21.304 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:28:21.304 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:28:21.305 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: mon.b calling monitor election 2024-09-17T08:28:21.305 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:21.305 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:28:21.305 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:28:21.305 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:28:21.305 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-17T08:28:21.305 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: monmap e2: 2 mons at {a=[v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0],b=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T08:28:21.305 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: fsmap 2024-09-17T08:28:21.305 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T08:28:21.305 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: mgrmap e13: y(active, since 40s) 2024-09-17T08:28:21.305 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: overall HEALTH_OK 2024-09-17T08:28:21.305 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:21.305 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:21.305 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:21.306 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:28:21.306 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:21.306 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:21.306 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: Deploying daemon mon.c on smithi118 2024-09-17T08:28:21.306 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: from='client.? 172.21.15.170:0/4211370730' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T08:28:21.306 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:28:21.306 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:21.306 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 ceph-mon[41418]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-17T08:28:21.306 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 bash[41380]: 4cd40efe8dc5fe4f1d8a866d0890747eb0a3776890e93ca540d9ba1a40f2f297 2024-09-17T08:28:21.306 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:21 smithi118 systemd[1]: Started Ceph mon.c for 7862d192-74ce-11ef-bceb-c7b262605968. 2024-09-17T08:28:26.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:26 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:28:26.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:26 smithi170 ceph-mon[33271]: mon.a calling monitor election 2024-09-17T08:28:26.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:26 smithi170 ceph-mon[33271]: mon.b calling monitor election 2024-09-17T08:28:26.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:26 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:28:26.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:26 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:28:26.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:26 smithi170 ceph-mon[33271]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:26.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:26 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:28:26.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:26 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:28:26.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:26 smithi170 ceph-mon[33271]: mon.c calling monitor election 2024-09-17T08:28:26.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:26 smithi170 ceph-mon[33271]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:26.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:26 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:28:26.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:26 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:28:26.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:26 smithi170 ceph-mon[33271]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:26.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:26 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:28:26.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:26 smithi170 ceph-mon[33271]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-17T08:28:26.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:26 smithi170 ceph-mon[33271]: monmap e3: 3 mons at {a=[v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0],b=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0],c=[v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T08:28:26.397 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:26 smithi170 ceph-mon[33271]: fsmap 2024-09-17T08:28:26.397 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:26 smithi170 ceph-mon[33271]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T08:28:26.397 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:26 smithi170 ceph-mon[33271]: mgrmap e13: y(active, since 48s) 2024-09-17T08:28:26.397 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:26 smithi170 ceph-mon[33271]: overall HEALTH_OK 2024-09-17T08:28:26.397 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:26 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:26.397 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:26 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:26.397 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:26 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:26.397 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:26 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:26.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:28:26.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[29491]: mon.a calling monitor election 2024-09-17T08:28:26.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[29491]: mon.b calling monitor election 2024-09-17T08:28:26.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:28:26.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:28:26.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[29491]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:26.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:28:26.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:28:26.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[29491]: mon.c calling monitor election 2024-09-17T08:28:26.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[29491]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:26.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:28:26.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:28:26.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[29491]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:26.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:28:26.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[29491]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-17T08:28:26.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[29491]: monmap e3: 3 mons at {a=[v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0],b=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0],c=[v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T08:28:26.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[29491]: fsmap 2024-09-17T08:28:26.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[29491]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T08:28:26.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[29491]: mgrmap e13: y(active, since 48s) 2024-09-17T08:28:26.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[29491]: overall HEALTH_OK 2024-09-17T08:28:26.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:26.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:26.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:26.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:26.529 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:28:26.529 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[41418]: mon.a calling monitor election 2024-09-17T08:28:26.529 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[41418]: mon.b calling monitor election 2024-09-17T08:28:26.529 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:28:26.529 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:28:26.529 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[41418]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:26.529 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:28:26.529 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:28:26.529 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[41418]: mon.c calling monitor election 2024-09-17T08:28:26.530 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[41418]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:26.530 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:28:26.530 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:28:26.530 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[41418]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:26.530 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:28:26.530 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[41418]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-17T08:28:26.530 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[41418]: monmap e3: 3 mons at {a=[v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0],b=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0],c=[v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T08:28:26.530 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[41418]: fsmap 2024-09-17T08:28:26.530 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[41418]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T08:28:26.530 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[41418]: mgrmap e13: y(active, since 48s) 2024-09-17T08:28:26.530 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[41418]: overall HEALTH_OK 2024-09-17T08:28:26.530 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:26.530 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:26.530 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:26.530 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:26 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:27.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:27 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:28:27.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:27 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:28:27.438 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-17T08:28:27.438 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":3,"fsid":"7862d192-74ce-11ef-bceb-c7b262605968","modified":"2024-09-17T08:28:21.027068Z","created":"2024-09-17T08:26:39.965247Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:3300","nonce":0},{"type":"v1","addr":"172.21.15.118:6789","nonce":0}]},"addr":"172.21.15.118:6789/0","public_addr":"172.21.15.118:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:3300","nonce":0},{"type":"v1","addr":"172.21.15.170:6789","nonce":0}]},"addr":"172.21.15.170:6789/0","public_addr":"172.21.15.170:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:3301","nonce":0},{"type":"v1","addr":"172.21.15.118:6790","nonce":0}]},"addr":"172.21.15.118:6790/0","public_addr":"172.21.15.118:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-17T08:28:27.439 INFO:teuthology.orchestra.run.smithi170.stderr:dumped monmap epoch 3 2024-09-17T08:28:27.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:27 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:28:27.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:27 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:28:27.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:27 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:28:27.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:27 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:28:28.116 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-17T08:28:28.116 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-17T08:28:28.387 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.387 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.387 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.387 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.387 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:28.387 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:28:28.388 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[29491]: Updating smithi118:/etc/ceph/ceph.conf 2024-09-17T08:28:28.388 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[29491]: Updating smithi170:/etc/ceph/ceph.conf 2024-09-17T08:28:28.388 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[29491]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:28.388 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.388 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[29491]: from='client.? 172.21.15.170:0/1919931115' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T08:28:28.388 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[29491]: Updating smithi118:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:28:28.388 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[29491]: Updating smithi170:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:28:28.388 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.388 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.388 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.388 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.388 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.388 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.389 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.389 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.389 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.389 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:28:28.389 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T08:28:28.389 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:28.389 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.389 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.389 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.389 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.389 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:28.389 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:28:28.390 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[41418]: Updating smithi118:/etc/ceph/ceph.conf 2024-09-17T08:28:28.390 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[41418]: Updating smithi170:/etc/ceph/ceph.conf 2024-09-17T08:28:28.390 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[41418]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:28.390 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.390 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[41418]: from='client.? 172.21.15.170:0/1919931115' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T08:28:28.390 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[41418]: Updating smithi118:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:28:28.390 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[41418]: Updating smithi170:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:28:28.390 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.390 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.390 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.390 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.390 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.390 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.390 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.390 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.391 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.391 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:28:28.391 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T08:28:28.391 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:28 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:28.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:28 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:28 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:28 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:28 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:28 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:28.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:28 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:28:28.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:28 smithi170 ceph-mon[33271]: Updating smithi118:/etc/ceph/ceph.conf 2024-09-17T08:28:28.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:28 smithi170 ceph-mon[33271]: Updating smithi170:/etc/ceph/ceph.conf 2024-09-17T08:28:28.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:28 smithi170 ceph-mon[33271]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:28.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:28 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:28 smithi170 ceph-mon[33271]: from='client.? 172.21.15.170:0/1919931115' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T08:28:28.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:28 smithi170 ceph-mon[33271]: Updating smithi118:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:28:28.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:28 smithi170 ceph-mon[33271]: Updating smithi170:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:28:28.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:28 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:28 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:28 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:28 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:28 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:28 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:28 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:28 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:28 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:28.397 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:28 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:28:28.397 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:28 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T08:28:28.397 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:28 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:28.718 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:28:29.508 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:29 smithi118 ceph-mon[29491]: Reconfiguring mon.a (unknown last config time)... 2024-09-17T08:28:29.508 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:29 smithi118 ceph-mon[29491]: Reconfiguring daemon mon.a on smithi118 2024-09-17T08:28:29.509 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:29 smithi118 ceph-mon[41418]: Reconfiguring mon.a (unknown last config time)... 2024-09-17T08:28:29.509 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:29 smithi118 ceph-mon[41418]: Reconfiguring daemon mon.a on smithi118 2024-09-17T08:28:29.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:29 smithi170 ceph-mon[33271]: Reconfiguring mon.a (unknown last config time)... 2024-09-17T08:28:29.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:29 smithi170 ceph-mon[33271]: Reconfiguring daemon mon.a on smithi118 2024-09-17T08:28:30.521 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:30 smithi118 ceph-mon[29491]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:30.522 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:30 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:30.522 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:30 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:30.522 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:30 smithi118 ceph-mon[29491]: Reconfiguring mon.c (monmap changed)... 2024-09-17T08:28:30.522 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:30 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:28:30.522 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:30 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T08:28:30.522 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:30 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:30.522 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:30 smithi118 ceph-mon[29491]: Reconfiguring daemon mon.c on smithi118 2024-09-17T08:28:30.522 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:30 smithi118 ceph-mon[41418]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:30.522 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:30 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:30.522 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:30 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:30.523 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:30 smithi118 ceph-mon[41418]: Reconfiguring mon.c (monmap changed)... 2024-09-17T08:28:30.523 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:30 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:28:30.523 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:30 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T08:28:30.523 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:30 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:30.523 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:30 smithi118 ceph-mon[41418]: Reconfiguring daemon mon.c on smithi118 2024-09-17T08:28:30.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:30 smithi170 ceph-mon[33271]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:30.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:30 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:30.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:30 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:30.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:30 smithi170 ceph-mon[33271]: Reconfiguring mon.c (monmap changed)... 2024-09-17T08:28:30.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:30 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:28:30.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:30 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T08:28:30.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:30 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:30.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:30 smithi170 ceph-mon[33271]: Reconfiguring daemon mon.c on smithi118 2024-09-17T08:28:32.056 INFO:teuthology.orchestra.run.smithi118.stdout:# minimal ceph.conf for 7862d192-74ce-11ef-bceb-c7b262605968 2024-09-17T08:28:32.057 INFO:teuthology.orchestra.run.smithi118.stdout:[global] 2024-09-17T08:28:32.057 INFO:teuthology.orchestra.run.smithi118.stdout: fsid = 7862d192-74ce-11ef-bceb-c7b262605968 2024-09-17T08:28:32.057 INFO:teuthology.orchestra.run.smithi118.stdout: mon_host = [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] 2024-09-17T08:28:32.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:32 smithi170 ceph-mon[33271]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:32.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:32 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:32.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:32 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:32.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:32 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:28:32.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:32 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T08:28:32.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:32 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:32.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:32 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/136176023' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:32.658 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:32 smithi118 ceph-mon[29491]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:32.658 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:32 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:32.658 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:32 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:32.658 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:32 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:28:32.658 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:32 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T08:28:32.658 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:32 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:32.658 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:32 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/136176023' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:32.658 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:32 smithi118 ceph-mon[41418]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:32.659 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:32 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:32.659 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:32 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:32.659 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:32 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:28:32.659 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:32 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T08:28:32.659 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:32 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:32.659 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:32 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/136176023' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:32.679 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-17T08:28:32.679 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-17T08:28:32.679 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/ceph/ceph.conf 2024-09-17T08:28:32.706 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-17T08:28:32.706 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:28:32.772 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-17T08:28:32.772 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/ceph.conf 2024-09-17T08:28:32.799 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-17T08:28:32.799 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:28:32.864 INFO:tasks.cephadm:Adding mgr.y on smithi118 2024-09-17T08:28:32.864 INFO:tasks.cephadm:Adding mgr.x on smithi170 2024-09-17T08:28:32.864 DEBUG:teuthology.orchestra.run.smithi170:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph orch apply mgr '2;smithi118=y;smithi170=x' 2024-09-17T08:28:33.100 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:28:33.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:33 smithi170 ceph-mon[33271]: Reconfiguring mon.b (monmap changed)... 2024-09-17T08:28:33.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:33 smithi170 ceph-mon[33271]: Reconfiguring daemon mon.b on smithi170 2024-09-17T08:28:33.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:33 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:33.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:33 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:33.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:33 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:28:33.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:33 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:33.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:33 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:28:33.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:33 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:33.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:33 smithi118 ceph-mon[29491]: Reconfiguring mon.b (monmap changed)... 2024-09-17T08:28:33.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:33 smithi118 ceph-mon[29491]: Reconfiguring daemon mon.b on smithi170 2024-09-17T08:28:33.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:33 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:33.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:33 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:33.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:33 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:28:33.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:33 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:33.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:33 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:28:33.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:33 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:33.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:33 smithi118 ceph-mon[41418]: Reconfiguring mon.b (monmap changed)... 2024-09-17T08:28:33.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:33 smithi118 ceph-mon[41418]: Reconfiguring daemon mon.b on smithi170 2024-09-17T08:28:33.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:33 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:33.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:33 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:33.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:33 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:28:33.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:33 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:33.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:33 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:28:33.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:33 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:34.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:34 smithi170 ceph-mon[33271]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:34.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:34 smithi118 ceph-mon[29491]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:34.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:34 smithi118 ceph-mon[41418]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:34.814 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled mgr update... 2024-09-17T08:28:35.434 DEBUG:teuthology.orchestra.run.smithi170:mgr.x> sudo journalctl -f -n 0 -u ceph-7862d192-74ce-11ef-bceb-c7b262605968@mgr.x.service 2024-09-17T08:28:35.436 INFO:tasks.cephadm:Deploying OSDs... 2024-09-17T08:28:35.437 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-17T08:28:35.437 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T08:28:35.456 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T08:28:35.457 DEBUG:teuthology.orchestra.run.smithi118:> stat /dev/vg_nvme/lv_1 2024-09-17T08:28:35.516 INFO:teuthology.orchestra.run.smithi118.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T08:28:35.516 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T08:28:35.516 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 5h/5d Inode: 961 Links: 1 2024-09-17T08:28:35.516 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T08:28:35.517 INFO:teuthology.orchestra.run.smithi118.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T08:28:35.517 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 2024-09-17 08:28:20.435015016 +0000 2024-09-17T08:28:35.517 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2024-09-17 08:23:54.500024848 +0000 2024-09-17T08:28:35.517 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2024-09-17 08:23:54.500024848 +0000 2024-09-17T08:28:35.517 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: 2024-09-17 08:23:54.500024848 +0000 2024-09-17T08:28:35.517 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T08:28:35.585 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records in 2024-09-17T08:28:35.585 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records out 2024-09-17T08:28:35.585 INFO:teuthology.orchestra.run.smithi118.stderr:512 bytes copied, 0.000130717 s, 3.9 MB/s 2024-09-17T08:28:35.586 DEBUG:teuthology.orchestra.run.smithi118:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T08:28:35.646 DEBUG:teuthology.orchestra.run.smithi118:> stat /dev/vg_nvme/lv_2 2024-09-17T08:28:35.704 INFO:teuthology.orchestra.run.smithi118.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T08:28:35.704 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T08:28:35.704 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 5h/5d Inode: 973 Links: 1 2024-09-17T08:28:35.704 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T08:28:35.704 INFO:teuthology.orchestra.run.smithi118.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T08:28:35.705 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 2024-09-17 08:28:20.435015016 +0000 2024-09-17T08:28:35.705 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2024-09-17 08:23:54.501024627 +0000 2024-09-17T08:28:35.705 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2024-09-17 08:23:54.501024627 +0000 2024-09-17T08:28:35.705 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: 2024-09-17 08:23:54.501024627 +0000 2024-09-17T08:28:35.705 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T08:28:35.770 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records in 2024-09-17T08:28:35.770 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records out 2024-09-17T08:28:35.770 INFO:teuthology.orchestra.run.smithi118.stderr:512 bytes copied, 0.000123534 s, 4.1 MB/s 2024-09-17T08:28:35.771 DEBUG:teuthology.orchestra.run.smithi118:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T08:28:35.829 DEBUG:teuthology.orchestra.run.smithi118:> stat /dev/vg_nvme/lv_3 2024-09-17T08:28:35.886 INFO:teuthology.orchestra.run.smithi118.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T08:28:35.886 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T08:28:35.886 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 5h/5d Inode: 976 Links: 1 2024-09-17T08:28:35.886 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T08:28:35.886 INFO:teuthology.orchestra.run.smithi118.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T08:28:35.886 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 2024-09-17 08:28:20.436014994 +0000 2024-09-17T08:28:35.886 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2024-09-17 08:23:54.503024186 +0000 2024-09-17T08:28:35.887 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2024-09-17 08:23:54.503024186 +0000 2024-09-17T08:28:35.887 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: 2024-09-17 08:23:54.503024186 +0000 2024-09-17T08:28:35.887 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T08:28:35.949 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:35 smithi118 ceph-mon[29491]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi118=y;smithi170=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:28:35.949 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:35 smithi118 ceph-mon[29491]: Saving service mgr spec with placement smithi118=y;smithi170=x;count:2 2024-09-17T08:28:35.949 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:35 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:35.949 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:35 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:28:35.949 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:35 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:35.949 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:35 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:28:35.949 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:35 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:35.949 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:35 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:28:35.949 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:35 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-17T08:28:35.950 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:35 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T08:28:35.950 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:35 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:35.950 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:35 smithi118 ceph-mon[29491]: Deploying daemon mgr.x on smithi170 2024-09-17T08:28:35.950 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:35 smithi118 ceph-mon[29491]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:35.950 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:35 smithi118 ceph-mon[41418]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi118=y;smithi170=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:28:35.950 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:35 smithi118 ceph-mon[41418]: Saving service mgr spec with placement smithi118=y;smithi170=x;count:2 2024-09-17T08:28:35.950 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:35 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:35.950 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:35 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:28:35.950 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:35 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:35.951 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:35 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:28:35.951 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:35 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:35.951 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:35 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:28:35.951 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:35 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-17T08:28:35.951 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:35 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T08:28:35.951 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:35 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:35.951 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:35 smithi118 ceph-mon[41418]: Deploying daemon mgr.x on smithi170 2024-09-17T08:28:35.951 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:35 smithi118 ceph-mon[41418]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:35.952 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records in 2024-09-17T08:28:35.952 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records out 2024-09-17T08:28:35.952 INFO:teuthology.orchestra.run.smithi118.stderr:512 bytes copied, 8.9424e-05 s, 5.7 MB/s 2024-09-17T08:28:35.953 DEBUG:teuthology.orchestra.run.smithi118:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T08:28:36.009 DEBUG:teuthology.orchestra.run.smithi118:> stat /dev/vg_nvme/lv_4 2024-09-17T08:28:36.065 INFO:teuthology.orchestra.run.smithi118.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T08:28:36.065 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T08:28:36.065 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 5h/5d Inode: 967 Links: 1 2024-09-17T08:28:36.065 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T08:28:36.065 INFO:teuthology.orchestra.run.smithi118.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T08:28:36.065 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 2024-09-17 08:28:20.436014994 +0000 2024-09-17T08:28:36.065 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2024-09-17 08:23:54.500024848 +0000 2024-09-17T08:28:36.065 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2024-09-17 08:23:54.500024848 +0000 2024-09-17T08:28:36.065 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: 2024-09-17 08:23:54.500024848 +0000 2024-09-17T08:28:36.066 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T08:28:36.128 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records in 2024-09-17T08:28:36.128 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records out 2024-09-17T08:28:36.128 INFO:teuthology.orchestra.run.smithi118.stderr:512 bytes copied, 0.000114467 s, 4.5 MB/s 2024-09-17T08:28:36.130 DEBUG:teuthology.orchestra.run.smithi118:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T08:28:36.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:35 smithi170 ceph-mon[33271]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi118=y;smithi170=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:28:36.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:35 smithi170 ceph-mon[33271]: Saving service mgr spec with placement smithi118=y;smithi170=x;count:2 2024-09-17T08:28:36.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:35 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:36.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:35 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:28:36.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:35 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:36.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:35 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:28:36.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:35 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:36.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:35 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:28:36.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:35 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-17T08:28:36.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:35 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T08:28:36.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:35 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:36.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:35 smithi170 ceph-mon[33271]: Deploying daemon mgr.x on smithi170 2024-09-17T08:28:36.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:35 smithi170 ceph-mon[33271]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:36.186 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-17T08:28:36.186 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T08:28:36.204 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T08:28:36.204 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_1 2024-09-17T08:28:36.260 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T08:28:36.260 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T08:28:36.260 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 980 Links: 1 2024-09-17T08:28:36.260 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T08:28:36.260 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T08:28:36.260 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-09-17 08:28:11.598674883 +0000 2024-09-17T08:28:36.260 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-09-17 08:23:55.278586658 +0000 2024-09-17T08:28:36.260 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-09-17 08:23:55.278586658 +0000 2024-09-17T08:28:36.260 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-09-17 08:23:55.278586658 +0000 2024-09-17T08:28:36.261 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T08:28:36.324 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-09-17T08:28:36.324 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-09-17T08:28:36.324 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.00011604 s, 4.4 MB/s 2024-09-17T08:28:36.326 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T08:28:36.382 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_2 2024-09-17T08:28:36.439 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T08:28:36.439 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T08:28:36.439 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 967 Links: 1 2024-09-17T08:28:36.439 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T08:28:36.440 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T08:28:36.440 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-09-17 08:28:11.598674883 +0000 2024-09-17T08:28:36.440 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-09-17 08:23:55.278586658 +0000 2024-09-17T08:28:36.440 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-09-17 08:23:55.278586658 +0000 2024-09-17T08:28:36.440 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-09-17 08:23:55.278586658 +0000 2024-09-17T08:28:36.441 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T08:28:36.502 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-09-17T08:28:36.503 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-09-17T08:28:36.503 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 8.6518e-05 s, 5.9 MB/s 2024-09-17T08:28:36.504 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T08:28:36.560 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_3 2024-09-17T08:28:36.616 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T08:28:36.616 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T08:28:36.616 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 973 Links: 1 2024-09-17T08:28:36.616 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T08:28:36.616 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T08:28:36.616 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-09-17 08:28:11.598674883 +0000 2024-09-17T08:28:36.617 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-09-17 08:23:55.278586658 +0000 2024-09-17T08:28:36.617 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-09-17 08:23:55.278586658 +0000 2024-09-17T08:28:36.617 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-09-17 08:23:55.278586658 +0000 2024-09-17T08:28:36.617 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T08:28:36.679 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-09-17T08:28:36.680 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-09-17T08:28:36.680 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000107936 s, 4.7 MB/s 2024-09-17T08:28:36.681 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T08:28:36.737 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_4 2024-09-17T08:28:36.793 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T08:28:36.793 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T08:28:36.793 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 970 Links: 1 2024-09-17T08:28:36.793 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T08:28:36.793 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T08:28:36.793 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-09-17 08:28:11.599674860 +0000 2024-09-17T08:28:36.793 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-09-17 08:23:55.278586658 +0000 2024-09-17T08:28:36.793 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-09-17 08:23:55.278586658 +0000 2024-09-17T08:28:36.793 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-09-17 08:23:55.278586658 +0000 2024-09-17T08:28:36.793 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T08:28:36.874 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-09-17T08:28:36.874 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-09-17T08:28:36.874 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 9.2947e-05 s, 5.5 MB/s 2024-09-17T08:28:36.875 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T08:28:36.931 INFO:tasks.cephadm:Deploying osd.0 on smithi118 with /dev/vg_nvme/lv_4... 2024-09-17T08:28:36.931 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-17T08:28:37.132 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:28:37.280 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:28:37 smithi170 systemd[1]: Starting Ceph mgr.x for 7862d192-74ce-11ef-bceb-c7b262605968... 2024-09-17T08:28:37.545 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:28:37 smithi170 podman[35336]: 2024-09-17 08:28:37.182576557 +0000 UTC m=+0.015440591 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T08:28:37.545 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:28:37 smithi170 podman[35336]: 2024-09-17 08:28:37.30555944 +0000 UTC m=+0.138423474 container create 2bd244aa4628633b716c2ea1d6347fcec992065db1b0958c2fa2761efe66bcf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x, RELEASE=reef-19e2a2b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True) 2024-09-17T08:28:37.546 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:28:37 smithi170 podman[35336]: 2024-09-17 08:28:37.516331897 +0000 UTC m=+0.349195934 container init 2bd244aa4628633b716c2ea1d6347fcec992065db1b0958c2fa2761efe66bcf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=) 2024-09-17T08:28:37.546 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:28:37 smithi170 podman[35336]: 2024-09-17 08:28:37.521156003 +0000 UTC m=+0.354020035 container start 2bd244aa4628633b716c2ea1d6347fcec992065db1b0958c2fa2761efe66bcf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-17T08:28:37.546 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:28:37 smithi170 ceph-mgr[35374]: -- 172.21.15.170:0/3671372045 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b61e642d00 con 0x55b61e617400 2024-09-17T08:28:37.868 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:28:37 smithi170 bash[35336]: 2bd244aa4628633b716c2ea1d6347fcec992065db1b0958c2fa2761efe66bcf5 2024-09-17T08:28:37.868 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:28:37 smithi170 systemd[1]: Started Ceph mgr.x for 7862d192-74ce-11ef-bceb-c7b262605968. 2024-09-17T08:28:37.868 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:28:37 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:28:37.642+0000 7f73e46cd040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T08:28:37.868 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:28:37 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:28:37.708+0000 7f73e46cd040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T08:28:37.868 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:28:37 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:28:37.867+0000 7f73e46cd040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T08:28:38.145 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:28:37 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:28:37.936+0000 7f73e46cd040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T08:28:38.145 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:28:38 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:28:38.114+0000 7f73e46cd040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:28:38.551 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:28:38 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:28:38.255+0000 7f73e46cd040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T08:28:38.552 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:38 smithi170 ceph-mon[33271]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:38.552 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:38 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:38.552 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:38 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:38.552 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:38 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:38.552 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:38 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:38.552 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:38 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:28:38.623 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:38 smithi118 ceph-mon[29491]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:38.624 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:38 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:38.624 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:38 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:38.624 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:38 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:38.624 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:38 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:38.624 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:38 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:28:38.624 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:38 smithi118 ceph-mon[41418]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:38.624 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:38 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:38.624 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:38 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:38.624 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:38 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:38.624 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:38 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:38.625 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:38 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:28:38.826 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:28:38 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:28:38.576+0000 7f73e46cd040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T08:28:38.827 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:28:38 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:28:38.636+0000 7f73e46cd040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T08:28:39.010 INFO:teuthology.orchestra.run.smithi118.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-17T08:28:39.094 INFO:teuthology.orchestra.run.smithi118.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-17T08:28:39.105 INFO:teuthology.orchestra.run.smithi118.stderr: stderr: 10+0 records in 2024-09-17T08:28:39.105 INFO:teuthology.orchestra.run.smithi118.stderr:10+0 records out 2024-09-17T08:28:39.105 INFO:teuthology.orchestra.run.smithi118.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00878076 s, 1.2 GB/s 2024-09-17T08:28:39.105 INFO:teuthology.orchestra.run.smithi118.stderr:--> Zapping successful for: 2024-09-17T08:28:39.145 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:28:38 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:28:38.826+0000 7f73e46cd040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T08:28:39.948 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:39 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:39.948 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:39 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:39.948 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:39 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:39.948 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:39 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:28:39.948 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:39 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:39.948 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:39 smithi118 ceph-mon[29491]: Reconfiguring mgr.y (unknown last config time)... 2024-09-17T08:28:39.948 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:39 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:28:39.948 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:39 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T08:28:39.948 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:39 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:39.948 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:39 smithi118 ceph-mon[29491]: Reconfiguring daemon mgr.y on smithi118 2024-09-17T08:28:39.948 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:39 smithi118 ceph-mon[29491]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:39.949 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:39 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:39.949 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:39 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:39.949 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:39 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:39.949 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:39 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:28:39.949 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:39 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:39.949 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:39 smithi118 ceph-mon[41418]: Reconfiguring mgr.y (unknown last config time)... 2024-09-17T08:28:39.949 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:39 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:28:39.949 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:39 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T08:28:39.949 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:39 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:39.949 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:39 smithi118 ceph-mon[41418]: Reconfiguring daemon mgr.y on smithi118 2024-09-17T08:28:39.949 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:39 smithi118 ceph-mon[41418]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:39.968 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi118:vg_nvme/lv_4 2024-09-17T08:28:40.145 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:28:39 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:28:39.687+0000 7f73e46cd040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T08:28:40.145 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:28:39 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:28:39.751+0000 7f73e46cd040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T08:28:40.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:39 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:40.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:39 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:40.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:39 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:40.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:39 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:28:40.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:39 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:40.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:39 smithi170 ceph-mon[33271]: Reconfiguring mgr.y (unknown last config time)... 2024-09-17T08:28:40.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:39 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:28:40.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:39 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T08:28:40.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:39 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:40.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:39 smithi170 ceph-mon[33271]: Reconfiguring daemon mgr.y on smithi118 2024-09-17T08:28:40.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:39 smithi170 ceph-mon[33271]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:40.715 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:28:40 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:28:40.460+0000 7f73e46cd040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T08:28:40.715 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:28:40 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:28:40.574+0000 7f73e46cd040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T08:28:40.715 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:28:40 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:28:40.714+0000 7f73e46cd040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:28:40.926 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:28:40.987 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:28:40 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:28:40.774+0000 7f73e46cd040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T08:28:40.987 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:28:40 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:28:40.920+0000 7f73e46cd040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T08:28:40.987 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:28:40 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:28:40.986+0000 7f73e46cd040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T08:28:41.262 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:28:41 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:28:41.044+0000 7f73e46cd040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T08:28:41.262 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:28:41 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:28:41.196+0000 7f73e46cd040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T08:28:41.262 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:28:41 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:28:41.260+0000 7f73e46cd040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T08:28:41.529 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:28:41 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:28:41.326+0000 7f73e46cd040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T08:28:41.529 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:28:41 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:28:41.410+0000 7f73e46cd040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T08:28:41.530 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:28:41 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:28:41.469+0000 7f73e46cd040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T08:28:41.530 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:28:41 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:28:41.528+0000 7f73e46cd040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T08:28:42.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:41 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:42.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:41 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:42.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:41 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:28:42.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:41 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:42.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:41 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:28:42.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:41 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:42.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:41 smithi118 ceph-mon[41418]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:42.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:41 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:42.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:41 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:42.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:41 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:28:42.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:41 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:42.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:41 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:28:42.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:41 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:42.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:41 smithi118 ceph-mon[29491]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:42.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:41 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:42.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:41 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:42.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:41 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:28:42.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:41 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:42.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:41 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:28:42.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:41 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:42.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:41 smithi170 ceph-mon[33271]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:42.645 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:28:42 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:28:42.293+0000 7f73e46cd040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T08:28:43.438 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:43 smithi118 ceph-mon[29491]: Standby manager daemon x started 2024-09-17T08:28:43.438 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:43 smithi118 ceph-mon[29491]: from='mgr.? 172.21.15.170:0/3138034454' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-17T08:28:43.438 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:43 smithi118 ceph-mon[29491]: from='mgr.? 172.21.15.170:0/3138034454' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T08:28:43.438 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:43 smithi118 ceph-mon[29491]: from='mgr.? 172.21.15.170:0/3138034454' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-17T08:28:43.438 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:43 smithi118 ceph-mon[29491]: from='mgr.? 172.21.15.170:0/3138034454' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T08:28:43.438 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:43 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:43.438 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:43 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:28:43.438 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:43 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:28:43.438 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:43 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:43.439 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:43 smithi118 ceph-mon[41418]: Standby manager daemon x started 2024-09-17T08:28:43.439 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:43 smithi118 ceph-mon[41418]: from='mgr.? 172.21.15.170:0/3138034454' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-17T08:28:43.439 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:43 smithi118 ceph-mon[41418]: from='mgr.? 172.21.15.170:0/3138034454' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T08:28:43.439 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:43 smithi118 ceph-mon[41418]: from='mgr.? 172.21.15.170:0/3138034454' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-17T08:28:43.439 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:43 smithi118 ceph-mon[41418]: from='mgr.? 172.21.15.170:0/3138034454' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T08:28:43.439 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:43 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:43.439 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:43 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:28:43.439 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:43 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:28:43.439 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:43 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:43.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:43 smithi170 ceph-mon[33271]: Standby manager daemon x started 2024-09-17T08:28:43.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:43 smithi170 ceph-mon[33271]: from='mgr.? 172.21.15.170:0/3138034454' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-17T08:28:43.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:43 smithi170 ceph-mon[33271]: from='mgr.? 172.21.15.170:0/3138034454' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T08:28:43.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:43 smithi170 ceph-mon[33271]: from='mgr.? 172.21.15.170:0/3138034454' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-17T08:28:43.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:43 smithi170 ceph-mon[33271]: from='mgr.? 172.21.15.170:0/3138034454' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T08:28:43.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:43 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:43.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:43 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:28:43.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:43 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:28:43.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:43 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:44.514 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:44 smithi118 ceph-mon[29491]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:28:44.515 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:44 smithi118 ceph-mon[29491]: mgrmap e14: y(active, since 65s), standbys: x 2024-09-17T08:28:44.515 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:44 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-17T08:28:44.515 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:44 smithi118 ceph-mon[29491]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:44.515 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:44 smithi118 ceph-mon[41418]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:28:44.515 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:44 smithi118 ceph-mon[41418]: mgrmap e14: y(active, since 65s), standbys: x 2024-09-17T08:28:44.515 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:44 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-17T08:28:44.515 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:44 smithi118 ceph-mon[41418]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:44.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:44 smithi170 ceph-mon[33271]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:28:44.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:44 smithi170 ceph-mon[33271]: mgrmap e14: y(active, since 65s), standbys: x 2024-09-17T08:28:44.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:44 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-17T08:28:44.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:44 smithi170 ceph-mon[33271]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:45.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:45 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/3223386740' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ea41bec-a258-4d78-be6b-0299675f1106"}]: dispatch 2024-09-17T08:28:45.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:45 smithi170 ceph-mon[33271]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ea41bec-a258-4d78-be6b-0299675f1106"}]: dispatch 2024-09-17T08:28:45.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:45 smithi170 ceph-mon[33271]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ea41bec-a258-4d78-be6b-0299675f1106"}]': finished 2024-09-17T08:28:45.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:45 smithi170 ceph-mon[33271]: osdmap e5: 1 total, 0 up, 1 in 2024-09-17T08:28:45.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:45 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:28:45.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:45 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3223386740' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ea41bec-a258-4d78-be6b-0299675f1106"}]: dispatch 2024-09-17T08:28:45.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:45 smithi118 ceph-mon[29491]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ea41bec-a258-4d78-be6b-0299675f1106"}]: dispatch 2024-09-17T08:28:45.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:45 smithi118 ceph-mon[29491]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ea41bec-a258-4d78-be6b-0299675f1106"}]': finished 2024-09-17T08:28:45.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:45 smithi118 ceph-mon[29491]: osdmap e5: 1 total, 0 up, 1 in 2024-09-17T08:28:45.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:45 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:28:45.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:45 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/3223386740' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ea41bec-a258-4d78-be6b-0299675f1106"}]: dispatch 2024-09-17T08:28:45.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:45 smithi118 ceph-mon[41418]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ea41bec-a258-4d78-be6b-0299675f1106"}]: dispatch 2024-09-17T08:28:45.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:45 smithi118 ceph-mon[41418]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ea41bec-a258-4d78-be6b-0299675f1106"}]': finished 2024-09-17T08:28:45.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:45 smithi118 ceph-mon[41418]: osdmap e5: 1 total, 0 up, 1 in 2024-09-17T08:28:45.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:45 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:28:46.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:46 smithi170 ceph-mon[33271]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:46.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:46 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/47178254' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:28:46.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:46 smithi118 ceph-mon[29491]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:46.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:46 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/47178254' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:28:46.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:46 smithi118 ceph-mon[41418]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:46.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:46 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/47178254' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:28:48.707 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:48 smithi118 ceph-mon[29491]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:48.707 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:48 smithi118 ceph-mon[41418]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:48.711 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:48 smithi170 ceph-mon[33271]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:50.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:50 smithi118 ceph-mon[41418]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:50.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:50 smithi118 ceph-mon[29491]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:50.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:50 smithi170 ceph-mon[33271]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:52.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:52 smithi118 ceph-mon[41418]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:52.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:52 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-17T08:28:52.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:52 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:52.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:52 smithi118 ceph-mon[41418]: Deploying daemon osd.0 on smithi118 2024-09-17T08:28:52.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:52 smithi118 ceph-mon[29491]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:52.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:52 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-17T08:28:52.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:52 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:52.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:52 smithi118 ceph-mon[29491]: Deploying daemon osd.0 on smithi118 2024-09-17T08:28:52.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:52 smithi170 ceph-mon[33271]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:52.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:52 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-17T08:28:52.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:52 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:52.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:52 smithi170 ceph-mon[33271]: Deploying daemon osd.0 on smithi118 2024-09-17T08:28:54.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:54 smithi118 ceph-mon[29491]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:54.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:54 smithi118 ceph-mon[41418]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:54.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:54 smithi170 ceph-mon[33271]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:56.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:56 smithi118 ceph-mon[29491]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:56.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:56 smithi118 ceph-mon[41418]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:56.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:56 smithi170 ceph-mon[33271]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:57.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:57 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:28:57.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:57 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:57.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:57 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:57.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:57 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:28:57.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:57 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:57.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:57 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:57.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:57 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:28:57.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:57 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:57.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:57 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:28:58.693 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:28:58 smithi118 ceph-mon[29491]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:58.693 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:28:58 smithi118 ceph-mon[41418]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:28:58.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:28:58 smithi170 ceph-mon[33271]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:29:00.139 INFO:teuthology.orchestra.run.smithi118.stdout:Created osd(s) 0 on host 'smithi118' 2024-09-17T08:29:00.511 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:00 smithi118 ceph-mon[41418]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:29:00.511 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:00 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:00.511 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:00 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:00.512 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:00 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:29:00.512 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:00 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:29:00.512 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:00 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:00.512 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:00 smithi118 ceph-mon[41418]: from='osd.0 [v2:172.21.15.118:6802/3097740994,v1:172.21.15.118:6803/3097740994]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-17T08:29:00.512 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:00 smithi118 ceph-mon[41418]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-17T08:29:00.512 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:00 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:29:00.512 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:00 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:00.512 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:00 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:00.513 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:00 smithi118 ceph-mon[29491]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:29:00.513 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:00 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:00.513 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:00 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:00.513 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:00 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:29:00.513 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:00 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:29:00.513 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:00 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:00.513 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:00 smithi118 ceph-mon[29491]: from='osd.0 [v2:172.21.15.118:6802/3097740994,v1:172.21.15.118:6803/3097740994]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-17T08:29:00.513 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:00 smithi118 ceph-mon[29491]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-17T08:29:00.513 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:00 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:29:00.513 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:00 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:00.513 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:00 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:00.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:00 smithi170 ceph-mon[33271]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:29:00.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:00 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:00.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:00 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:00.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:00 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:29:00.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:00 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:29:00.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:00 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:00.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:00 smithi170 ceph-mon[33271]: from='osd.0 [v2:172.21.15.118:6802/3097740994,v1:172.21.15.118:6803/3097740994]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-17T08:29:00.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:00 smithi170 ceph-mon[33271]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-17T08:29:00.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:00 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:29:00.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:00 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:00.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:00 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:00.823 DEBUG:teuthology.orchestra.run.smithi118:osd.0> sudo journalctl -f -n 0 -u ceph-7862d192-74ce-11ef-bceb-c7b262605968@osd.0.service 2024-09-17T08:29:00.826 INFO:tasks.cephadm:Deploying osd.1 on smithi118 with /dev/vg_nvme/lv_3... 2024-09-17T08:29:00.826 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-17T08:29:01.638 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:29:01.733 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:01 smithi118 ceph-mon[29491]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-17T08:29:01.733 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:01 smithi118 ceph-mon[29491]: osdmap e6: 1 total, 0 up, 1 in 2024-09-17T08:29:01.734 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:01 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:29:01.734 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:01 smithi118 ceph-mon[29491]: from='osd.0 [v2:172.21.15.118:6802/3097740994,v1:172.21.15.118:6803/3097740994]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-17T08:29:01.734 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:01 smithi118 ceph-mon[29491]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-17T08:29:01.734 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:01 smithi118 ceph-mon[41418]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-17T08:29:01.734 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:01 smithi118 ceph-mon[41418]: osdmap e6: 1 total, 0 up, 1 in 2024-09-17T08:29:01.734 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:01 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:29:01.734 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:01 smithi118 ceph-mon[41418]: from='osd.0 [v2:172.21.15.118:6802/3097740994,v1:172.21.15.118:6803/3097740994]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-17T08:29:01.734 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:01 smithi118 ceph-mon[41418]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-17T08:29:01.735 INFO:journalctl@ceph.osd.0.smithi118.stdout:Sep 17 08:29:01 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-0[45818]: 2024-09-17T08:29:01.418+0000 7fd48f3c5640 -1 osd.0 0 waiting for initial osdmap 2024-09-17T08:29:01.735 INFO:journalctl@ceph.osd.0.smithi118.stdout:Sep 17 08:29:01 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-0[45818]: 2024-09-17T08:29:01.420+0000 7fd48b1ee640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T08:29:01.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:01 smithi170 ceph-mon[33271]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-17T08:29:01.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:01 smithi170 ceph-mon[33271]: osdmap e6: 1 total, 0 up, 1 in 2024-09-17T08:29:01.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:01 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:29:01.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:01 smithi170 ceph-mon[33271]: from='osd.0 [v2:172.21.15.118:6802/3097740994,v1:172.21.15.118:6803/3097740994]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-17T08:29:01.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:01 smithi170 ceph-mon[33271]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-17T08:29:02.547 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:02 smithi118 ceph-mon[29491]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:29:02.547 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:02 smithi118 ceph-mon[29491]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2024-09-17T08:29:02.547 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:02 smithi118 ceph-mon[29491]: osdmap e7: 1 total, 0 up, 1 in 2024-09-17T08:29:02.547 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:02 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:29:02.547 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:02 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:29:02.547 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:02 smithi118 ceph-mon[41418]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:29:02.548 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:02 smithi118 ceph-mon[41418]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2024-09-17T08:29:02.548 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:02 smithi118 ceph-mon[41418]: osdmap e7: 1 total, 0 up, 1 in 2024-09-17T08:29:02.548 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:02 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:29:02.548 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:02 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:29:02.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:02 smithi170 ceph-mon[33271]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:29:02.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:02 smithi170 ceph-mon[33271]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2024-09-17T08:29:02.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:02 smithi170 ceph-mon[33271]: osdmap e7: 1 total, 0 up, 1 in 2024-09-17T08:29:02.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:02 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:29:02.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:02 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:29:03.583 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:03 smithi118 ceph-mon[41418]: purged_snaps scrub starts 2024-09-17T08:29:03.583 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:03 smithi118 ceph-mon[41418]: purged_snaps scrub ok 2024-09-17T08:29:03.583 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:03 smithi118 ceph-mon[41418]: osd.0 [v2:172.21.15.118:6802/3097740994,v1:172.21.15.118:6803/3097740994] boot 2024-09-17T08:29:03.583 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:03 smithi118 ceph-mon[41418]: osdmap e8: 1 total, 1 up, 1 in 2024-09-17T08:29:03.583 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:03 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:29:03.584 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:03 smithi118 ceph-mon[29491]: purged_snaps scrub starts 2024-09-17T08:29:03.584 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:03 smithi118 ceph-mon[29491]: purged_snaps scrub ok 2024-09-17T08:29:03.584 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:03 smithi118 ceph-mon[29491]: osd.0 [v2:172.21.15.118:6802/3097740994,v1:172.21.15.118:6803/3097740994] boot 2024-09-17T08:29:03.584 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:03 smithi118 ceph-mon[29491]: osdmap e8: 1 total, 1 up, 1 in 2024-09-17T08:29:03.584 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:03 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:29:03.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:03 smithi170 ceph-mon[33271]: purged_snaps scrub starts 2024-09-17T08:29:03.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:03 smithi170 ceph-mon[33271]: purged_snaps scrub ok 2024-09-17T08:29:03.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:03 smithi170 ceph-mon[33271]: osd.0 [v2:172.21.15.118:6802/3097740994,v1:172.21.15.118:6803/3097740994] boot 2024-09-17T08:29:03.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:03 smithi170 ceph-mon[33271]: osdmap e8: 1 total, 1 up, 1 in 2024-09-17T08:29:03.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:03 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:29:04.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:04 smithi118 ceph-mon[29491]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:04.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:04 smithi118 ceph-mon[29491]: osdmap e9: 1 total, 1 up, 1 in 2024-09-17T08:29:04.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:04 smithi118 ceph-mon[41418]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:04.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:04 smithi118 ceph-mon[41418]: osdmap e9: 1 total, 1 up, 1 in 2024-09-17T08:29:04.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:04 smithi170 ceph-mon[33271]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:04.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:04 smithi170 ceph-mon[33271]: osdmap e9: 1 total, 1 up, 1 in 2024-09-17T08:29:05.073 INFO:teuthology.orchestra.run.smithi118.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-17T08:29:05.153 INFO:teuthology.orchestra.run.smithi118.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-17T08:29:05.163 INFO:teuthology.orchestra.run.smithi118.stderr: stderr: 10+0 records in 2024-09-17T08:29:05.163 INFO:teuthology.orchestra.run.smithi118.stderr:10+0 records out 2024-09-17T08:29:05.163 INFO:teuthology.orchestra.run.smithi118.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00878086 s, 1.2 GB/s 2024-09-17T08:29:05.163 INFO:teuthology.orchestra.run.smithi118.stderr:--> Zapping successful for: 2024-09-17T08:29:05.812 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:05 smithi118 ceph-mon[29491]: Detected new or changed devices on smithi118 2024-09-17T08:29:05.813 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:05 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:05.813 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:05 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:05.813 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:05 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:29:05.813 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:05 smithi118 ceph-mon[29491]: Adjusting osd_memory_target on smithi118 to 16064M 2024-09-17T08:29:05.813 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:05 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:05.813 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:05 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:29:05.813 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:05 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:29:05.813 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:05 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:05.813 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:05 smithi118 ceph-mon[29491]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:05.813 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:05 smithi118 ceph-mon[41418]: Detected new or changed devices on smithi118 2024-09-17T08:29:05.813 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:05 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:05.813 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:05 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:05.814 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:05 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:29:05.814 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:05 smithi118 ceph-mon[41418]: Adjusting osd_memory_target on smithi118 to 16064M 2024-09-17T08:29:05.814 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:05 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:05.814 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:05 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:29:05.814 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:05 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:29:05.814 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:05 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:05.814 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:05 smithi118 ceph-mon[41418]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:05.836 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi118:vg_nvme/lv_3 2024-09-17T08:29:06.032 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:29:06.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:05 smithi170 ceph-mon[33271]: Detected new or changed devices on smithi118 2024-09-17T08:29:06.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:05 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:06.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:05 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:06.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:05 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:29:06.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:05 smithi170 ceph-mon[33271]: Adjusting osd_memory_target on smithi118 to 16064M 2024-09-17T08:29:06.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:05 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:06.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:05 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:29:06.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:05 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:29:06.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:05 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:06.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:05 smithi170 ceph-mon[33271]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:08.498 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:08 smithi118 ceph-mon[29491]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:08.498 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:08 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:29:08.499 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:08 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:29:08.499 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:08 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:29:08.499 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:08 smithi118 ceph-mon[41418]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:08.499 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:08 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:29:08.499 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:08 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:29:08.499 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:08 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:29:08.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:08 smithi170 ceph-mon[33271]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:08.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:08 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:29:08.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:08 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:29:08.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:08 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:29:09.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:09 smithi118 ceph-mon[29491]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:29:09.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:09 smithi118 ceph-mon[41418]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:29:09.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:09 smithi170 ceph-mon[33271]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:29:10.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:10 smithi118 ceph-mon[29491]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:10.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:10 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/1849755863' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56073858-b245-4470-9941-1cdf67a69ec5"}]: dispatch 2024-09-17T08:29:10.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:10 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/1849755863' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56073858-b245-4470-9941-1cdf67a69ec5"}]': finished 2024-09-17T08:29:10.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:10 smithi118 ceph-mon[29491]: osdmap e10: 2 total, 1 up, 2 in 2024-09-17T08:29:10.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:10 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:29:10.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:10 smithi118 ceph-mon[41418]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:10.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:10 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/1849755863' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56073858-b245-4470-9941-1cdf67a69ec5"}]: dispatch 2024-09-17T08:29:10.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:10 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/1849755863' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56073858-b245-4470-9941-1cdf67a69ec5"}]': finished 2024-09-17T08:29:10.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:10 smithi118 ceph-mon[41418]: osdmap e10: 2 total, 1 up, 2 in 2024-09-17T08:29:10.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:10 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:29:10.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:10 smithi170 ceph-mon[33271]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:10.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:10 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/1849755863' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56073858-b245-4470-9941-1cdf67a69ec5"}]: dispatch 2024-09-17T08:29:10.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:10 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/1849755863' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56073858-b245-4470-9941-1cdf67a69ec5"}]': finished 2024-09-17T08:29:10.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:10 smithi170 ceph-mon[33271]: osdmap e10: 2 total, 1 up, 2 in 2024-09-17T08:29:10.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:10 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:29:11.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:11 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/1791234792' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:29:11.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:11 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/1791234792' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:29:11.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:11 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/1791234792' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:29:12.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:12 smithi118 ceph-mon[29491]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:12.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:12 smithi118 ceph-mon[41418]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:12.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:12 smithi170 ceph-mon[33271]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:14.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:14 smithi118 ceph-mon[29491]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:14.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:14 smithi118 ceph-mon[41418]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:14.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:14 smithi170 ceph-mon[33271]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:16.702 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:16 smithi118 ceph-mon[29491]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:16.702 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:16 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-17T08:29:16.702 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:16 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:29:16.702 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:16 smithi118 ceph-mon[41418]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:16.702 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:16 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-17T08:29:16.702 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:16 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:29:16.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:16 smithi170 ceph-mon[33271]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:16.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:16 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-17T08:29:16.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:16 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:29:17.700 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:17 smithi118 ceph-mon[29491]: Deploying daemon osd.1 on smithi118 2024-09-17T08:29:17.700 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:17 smithi118 ceph-mon[41418]: Deploying daemon osd.1 on smithi118 2024-09-17T08:29:17.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:17 smithi170 ceph-mon[33271]: Deploying daemon osd.1 on smithi118 2024-09-17T08:29:18.649 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:18 smithi118 ceph-mon[29491]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:18.649 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:18 smithi118 ceph-mon[41418]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:18.796 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:18 smithi170 ceph-mon[33271]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:20.731 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:20 smithi118 ceph-mon[29491]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:20.731 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:20 smithi118 ceph-mon[41418]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:20.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:20 smithi170 ceph-mon[33271]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:22.715 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:22 smithi118 ceph-mon[29491]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:22.715 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:22 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:29:22.715 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:22 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:22.715 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:22 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:22.715 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:22 smithi118 ceph-mon[41418]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:22.715 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:22 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:29:22.715 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:22 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:22.715 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:22 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:22.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:22 smithi170 ceph-mon[33271]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:22.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:22 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:29:22.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:22 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:22.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:22 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:24.737 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:24 smithi118 ceph-mon[29491]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:24.737 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:24 smithi118 ceph-mon[41418]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:24.839 INFO:teuthology.orchestra.run.smithi118.stdout:Created osd(s) 1 on host 'smithi118' 2024-09-17T08:29:24.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:24 smithi170 ceph-mon[33271]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:25.487 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:25 smithi118 ceph-mon[29491]: from='osd.1 [v2:172.21.15.118:6810/3447428990,v1:172.21.15.118:6811/3447428990]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-17T08:29:25.487 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:25 smithi118 ceph-mon[29491]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-17T08:29:25.487 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:25 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:25.487 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:25 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:25.487 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:25 smithi118 ceph-mon[41418]: from='osd.1 [v2:172.21.15.118:6810/3447428990,v1:172.21.15.118:6811/3447428990]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-17T08:29:25.487 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:25 smithi118 ceph-mon[41418]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-17T08:29:25.487 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:25 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:25.487 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:25 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:25.507 DEBUG:teuthology.orchestra.run.smithi118:osd.1> sudo journalctl -f -n 0 -u ceph-7862d192-74ce-11ef-bceb-c7b262605968@osd.1.service 2024-09-17T08:29:25.510 INFO:tasks.cephadm:Deploying osd.2 on smithi118 with /dev/vg_nvme/lv_2... 2024-09-17T08:29:25.510 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-17T08:29:25.751 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:29:25.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:25 smithi170 ceph-mon[33271]: from='osd.1 [v2:172.21.15.118:6810/3447428990,v1:172.21.15.118:6811/3447428990]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-17T08:29:25.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:25 smithi170 ceph-mon[33271]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-17T08:29:25.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:25 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:25.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:25 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:26.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:26 smithi118 ceph-mon[29491]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:26.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:26 smithi118 ceph-mon[29491]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-17T08:29:26.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:26 smithi118 ceph-mon[29491]: osdmap e11: 2 total, 1 up, 2 in 2024-09-17T08:29:26.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:26 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:29:26.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:26 smithi118 ceph-mon[29491]: from='osd.1 [v2:172.21.15.118:6810/3447428990,v1:172.21.15.118:6811/3447428990]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-17T08:29:26.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:26 smithi118 ceph-mon[29491]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-17T08:29:26.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:26 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:26.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:26 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:26.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:26 smithi118 ceph-mon[41418]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:26.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:26 smithi118 ceph-mon[41418]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-17T08:29:26.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:26 smithi118 ceph-mon[41418]: osdmap e11: 2 total, 1 up, 2 in 2024-09-17T08:29:26.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:26 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:29:26.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:26 smithi118 ceph-mon[41418]: from='osd.1 [v2:172.21.15.118:6810/3447428990,v1:172.21.15.118:6811/3447428990]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-17T08:29:26.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:26 smithi118 ceph-mon[41418]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-17T08:29:26.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:26 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:26.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:26 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:26.529 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 17 08:29:26 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-1[51742]: 2024-09-17T08:29:26.437+0000 7f0adecf9640 -1 osd.1 0 waiting for initial osdmap 2024-09-17T08:29:26.529 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 17 08:29:26 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-1[51742]: 2024-09-17T08:29:26.440+0000 7f0adab22640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T08:29:26.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:26 smithi170 ceph-mon[33271]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:26.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:26 smithi170 ceph-mon[33271]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-17T08:29:26.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:26 smithi170 ceph-mon[33271]: osdmap e11: 2 total, 1 up, 2 in 2024-09-17T08:29:26.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:26 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:29:26.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:26 smithi170 ceph-mon[33271]: from='osd.1 [v2:172.21.15.118:6810/3447428990,v1:172.21.15.118:6811/3447428990]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-17T08:29:26.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:26 smithi170 ceph-mon[33271]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-17T08:29:26.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:26 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:26.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:26 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:27.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:27 smithi118 ceph-mon[29491]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2024-09-17T08:29:27.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:27 smithi118 ceph-mon[29491]: osdmap e12: 2 total, 1 up, 2 in 2024-09-17T08:29:27.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:27 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:29:27.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:27 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:29:27.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:27 smithi118 ceph-mon[41418]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2024-09-17T08:29:27.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:27 smithi118 ceph-mon[41418]: osdmap e12: 2 total, 1 up, 2 in 2024-09-17T08:29:27.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:27 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:29:27.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:27 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:29:27.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:27 smithi170 ceph-mon[33271]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2024-09-17T08:29:27.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:27 smithi170 ceph-mon[33271]: osdmap e12: 2 total, 1 up, 2 in 2024-09-17T08:29:27.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:27 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:29:27.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:27 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:29:28.028 INFO:teuthology.orchestra.run.smithi118.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-17T08:29:28.122 INFO:teuthology.orchestra.run.smithi118.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-17T08:29:28.134 INFO:teuthology.orchestra.run.smithi118.stderr: stderr: 10+0 records in 2024-09-17T08:29:28.134 INFO:teuthology.orchestra.run.smithi118.stderr:10+0 records out 2024-09-17T08:29:28.134 INFO:teuthology.orchestra.run.smithi118.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00989533 s, 1.1 GB/s 2024-09-17T08:29:28.135 INFO:teuthology.orchestra.run.smithi118.stderr:--> Zapping successful for: 2024-09-17T08:29:28.698 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:28 smithi118 ceph-mon[29491]: purged_snaps scrub starts 2024-09-17T08:29:28.698 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:28 smithi118 ceph-mon[29491]: purged_snaps scrub ok 2024-09-17T08:29:28.698 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:28 smithi118 ceph-mon[29491]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:28.698 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:28 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:29:28.698 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:28 smithi118 ceph-mon[29491]: osd.1 [v2:172.21.15.118:6810/3447428990,v1:172.21.15.118:6811/3447428990] boot 2024-09-17T08:29:28.698 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:28 smithi118 ceph-mon[29491]: osdmap e13: 2 total, 2 up, 2 in 2024-09-17T08:29:28.698 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:28 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:29:28.699 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:28 smithi118 ceph-mon[41418]: purged_snaps scrub starts 2024-09-17T08:29:28.699 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:28 smithi118 ceph-mon[41418]: purged_snaps scrub ok 2024-09-17T08:29:28.699 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:28 smithi118 ceph-mon[41418]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:28.699 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:28 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:29:28.699 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:28 smithi118 ceph-mon[41418]: osd.1 [v2:172.21.15.118:6810/3447428990,v1:172.21.15.118:6811/3447428990] boot 2024-09-17T08:29:28.699 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:28 smithi118 ceph-mon[41418]: osdmap e13: 2 total, 2 up, 2 in 2024-09-17T08:29:28.699 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:28 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:29:28.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:28 smithi170 ceph-mon[33271]: purged_snaps scrub starts 2024-09-17T08:29:28.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:28 smithi170 ceph-mon[33271]: purged_snaps scrub ok 2024-09-17T08:29:28.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:28 smithi170 ceph-mon[33271]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:29:28.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:28 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:29:28.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:28 smithi170 ceph-mon[33271]: osd.1 [v2:172.21.15.118:6810/3447428990,v1:172.21.15.118:6811/3447428990] boot 2024-09-17T08:29:28.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:28 smithi170 ceph-mon[33271]: osdmap e13: 2 total, 2 up, 2 in 2024-09-17T08:29:28.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:28 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:29:29.599 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi118:vg_nvme/lv_2 2024-09-17T08:29:29.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:29 smithi118 ceph-mon[29491]: osdmap e14: 2 total, 2 up, 2 in 2024-09-17T08:29:29.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:29 smithi118 ceph-mon[41418]: osdmap e14: 2 total, 2 up, 2 in 2024-09-17T08:29:29.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:29 smithi170 ceph-mon[33271]: osdmap e14: 2 total, 2 up, 2 in 2024-09-17T08:29:30.096 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:29:30.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:30 smithi118 ceph-mon[29491]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:30.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:30 smithi118 ceph-mon[41418]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:30.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:30 smithi170 ceph-mon[33271]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:32.544 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:32 smithi118 ceph-mon[29491]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:32.544 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:32 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:32.544 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:32 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:32.545 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:32 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:29:32.547 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:32 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:29:32.548 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:32 smithi118 ceph-mon[41418]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:32.548 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:32 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:32.548 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:32 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:32.548 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:32 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:29:32.548 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:32 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:29:32.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:32 smithi170 ceph-mon[33271]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:32.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:32 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:32.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:32 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:32.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:32 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:29:32.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:32 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:29:33.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:33 smithi118 ceph-mon[41418]: Detected new or changed devices on smithi118 2024-09-17T08:29:33.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:33 smithi118 ceph-mon[41418]: Adjusting osd_memory_target on smithi118 to 8032M 2024-09-17T08:29:33.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:33 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:33.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:33 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:29:33.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:33 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:29:33.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:33 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:33.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:33 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:29:33.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:33 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:29:33.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:33 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:29:33.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:33 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:33.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:33 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:29:33.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:33 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:29:33.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:33 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:29:33.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:33 smithi118 ceph-mon[29491]: Detected new or changed devices on smithi118 2024-09-17T08:29:33.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:33 smithi118 ceph-mon[29491]: Adjusting osd_memory_target on smithi118 to 8032M 2024-09-17T08:29:33.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:33 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:33.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:33 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:29:33.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:33 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:29:33.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:33 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:33.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:33 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:29:33.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:33 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:29:33.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:33 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:29:33.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:33 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:33.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:33 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:29:33.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:33 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:29:33.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:33 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:29:33.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:33 smithi170 ceph-mon[33271]: Detected new or changed devices on smithi118 2024-09-17T08:29:33.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:33 smithi170 ceph-mon[33271]: Adjusting osd_memory_target on smithi118 to 8032M 2024-09-17T08:29:33.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:33 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:33.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:33 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:29:33.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:33 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:29:33.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:33 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:33.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:33 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:29:33.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:33 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:29:33.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:33 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:29:33.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:33 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:33.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:33 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:29:33.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:33 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:29:33.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:33 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:29:34.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:34 smithi170 ceph-mon[33271]: from='client.24170 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:29:34.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:34 smithi170 ceph-mon[33271]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:34.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:34 smithi118 ceph-mon[29491]: from='client.24170 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:29:34.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:34 smithi118 ceph-mon[29491]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:34.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:34 smithi118 ceph-mon[41418]: from='client.24170 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:29:34.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:34 smithi118 ceph-mon[41418]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:35.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:35 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3054644963' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "edce62b8-f6c0-4c4a-b789-45304fa5eeb4"}]: dispatch 2024-09-17T08:29:35.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:35 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3054644963' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "edce62b8-f6c0-4c4a-b789-45304fa5eeb4"}]': finished 2024-09-17T08:29:35.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:35 smithi118 ceph-mon[29491]: osdmap e15: 3 total, 2 up, 3 in 2024-09-17T08:29:35.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:35 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:29:35.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:35 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/3054644963' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "edce62b8-f6c0-4c4a-b789-45304fa5eeb4"}]: dispatch 2024-09-17T08:29:35.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:35 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/3054644963' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "edce62b8-f6c0-4c4a-b789-45304fa5eeb4"}]': finished 2024-09-17T08:29:35.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:35 smithi118 ceph-mon[41418]: osdmap e15: 3 total, 2 up, 3 in 2024-09-17T08:29:35.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:35 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:29:35.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:35 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/3054644963' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "edce62b8-f6c0-4c4a-b789-45304fa5eeb4"}]: dispatch 2024-09-17T08:29:35.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:35 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/3054644963' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "edce62b8-f6c0-4c4a-b789-45304fa5eeb4"}]': finished 2024-09-17T08:29:35.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:35 smithi170 ceph-mon[33271]: osdmap e15: 3 total, 2 up, 3 in 2024-09-17T08:29:35.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:35 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:29:36.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:36 smithi170 ceph-mon[33271]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:36.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:36 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/1286754083' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:29:37.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:36 smithi118 ceph-mon[29491]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:37.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:36 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/1286754083' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:29:37.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:36 smithi118 ceph-mon[41418]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:37.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:36 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/1286754083' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:29:38.719 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:38 smithi118 ceph-mon[29491]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:38.720 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:38 smithi118 ceph-mon[41418]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:38.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:38 smithi170 ceph-mon[33271]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:40.660 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:40 smithi118 ceph-mon[29491]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:40.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:40 smithi118 ceph-mon[41418]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:40.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:40 smithi170 ceph-mon[33271]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:42.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:42 smithi118 ceph-mon[29491]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:42.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:42 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-17T08:29:42.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:42 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:29:42.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:42 smithi118 ceph-mon[41418]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:42.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:42 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-17T08:29:42.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:42 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:29:42.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:42 smithi170 ceph-mon[33271]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:42.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:42 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-17T08:29:42.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:42 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:29:43.767 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:43 smithi118 ceph-mon[29491]: Deploying daemon osd.2 on smithi118 2024-09-17T08:29:43.768 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:43 smithi118 ceph-mon[41418]: Deploying daemon osd.2 on smithi118 2024-09-17T08:29:43.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:43 smithi170 ceph-mon[33271]: Deploying daemon osd.2 on smithi118 2024-09-17T08:29:44.712 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:44 smithi118 ceph-mon[29491]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:44.713 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:44 smithi118 ceph-mon[41418]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:44.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:44 smithi170 ceph-mon[33271]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:46.688 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:46 smithi118 ceph-mon[29491]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:46.689 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:46 smithi118 ceph-mon[41418]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:46.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:46 smithi170 ceph-mon[33271]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:47.588 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:47 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:29:47.588 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:47 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:47.588 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:47 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:47.588 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:47 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:29:47.589 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:47 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:47.589 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:47 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:47.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:47 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:29:47.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:47 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:47.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:47 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:48.829 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:48 smithi118 ceph-mon[29491]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:48.829 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:48 smithi118 ceph-mon[41418]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:48.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:48 smithi170 ceph-mon[33271]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:50.365 INFO:teuthology.orchestra.run.smithi118.stdout:Created osd(s) 2 on host 'smithi118' 2024-09-17T08:29:50.664 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:50 smithi118 ceph-mon[29491]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:50.665 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:50 smithi118 ceph-mon[29491]: from='osd.2 [v2:172.21.15.118:6818/1717873621,v1:172.21.15.118:6819/1717873621]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-17T08:29:50.665 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:50 smithi118 ceph-mon[29491]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-17T08:29:50.665 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:50 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:50.665 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:50 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:50.665 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:50 smithi118 ceph-mon[41418]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:50.665 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:50 smithi118 ceph-mon[41418]: from='osd.2 [v2:172.21.15.118:6818/1717873621,v1:172.21.15.118:6819/1717873621]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-17T08:29:50.665 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:50 smithi118 ceph-mon[41418]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-17T08:29:50.665 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:50 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:50.665 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:50 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:50.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:50 smithi170 ceph-mon[33271]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:50.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:50 smithi170 ceph-mon[33271]: from='osd.2 [v2:172.21.15.118:6818/1717873621,v1:172.21.15.118:6819/1717873621]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-17T08:29:50.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:50 smithi170 ceph-mon[33271]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-17T08:29:50.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:50 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:50.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:50 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:50.979 DEBUG:teuthology.orchestra.run.smithi118:osd.2> sudo journalctl -f -n 0 -u ceph-7862d192-74ce-11ef-bceb-c7b262605968@osd.2.service 2024-09-17T08:29:50.985 INFO:tasks.cephadm:Deploying osd.3 on smithi118 with /dev/vg_nvme/lv_1... 2024-09-17T08:29:50.985 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-17T08:29:51.230 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:29:51.694 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:51 smithi118 ceph-mon[29491]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-17T08:29:51.694 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:51 smithi118 ceph-mon[29491]: osdmap e16: 3 total, 2 up, 3 in 2024-09-17T08:29:51.694 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:51 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:29:51.694 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:51 smithi118 ceph-mon[29491]: from='osd.2 [v2:172.21.15.118:6818/1717873621,v1:172.21.15.118:6819/1717873621]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-17T08:29:51.694 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:51 smithi118 ceph-mon[29491]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-17T08:29:51.694 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:51 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:51.694 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:51 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:51.695 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:51 smithi118 ceph-mon[41418]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-17T08:29:51.695 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:51 smithi118 ceph-mon[41418]: osdmap e16: 3 total, 2 up, 3 in 2024-09-17T08:29:51.695 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:51 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:29:51.695 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:51 smithi118 ceph-mon[41418]: from='osd.2 [v2:172.21.15.118:6818/1717873621,v1:172.21.15.118:6819/1717873621]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-17T08:29:51.695 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:51 smithi118 ceph-mon[41418]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-17T08:29:51.695 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:51 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:51.695 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:51 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:51.695 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 17 08:29:51 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-2[57824]: 2024-09-17T08:29:51.574+0000 7f39cb422640 -1 osd.2 0 waiting for initial osdmap 2024-09-17T08:29:51.696 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 17 08:29:51 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-2[57824]: 2024-09-17T08:29:51.578+0000 7f39c724b640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T08:29:51.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:51 smithi170 ceph-mon[33271]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-17T08:29:51.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:51 smithi170 ceph-mon[33271]: osdmap e16: 3 total, 2 up, 3 in 2024-09-17T08:29:51.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:51 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:29:51.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:51 smithi170 ceph-mon[33271]: from='osd.2 [v2:172.21.15.118:6818/1717873621,v1:172.21.15.118:6819/1717873621]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-17T08:29:51.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:51 smithi170 ceph-mon[33271]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-17T08:29:51.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:51 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:51.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:51 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:52.683 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:52 smithi118 ceph-mon[41418]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:52.683 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:52 smithi118 ceph-mon[41418]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2024-09-17T08:29:52.683 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:52 smithi118 ceph-mon[41418]: osdmap e17: 3 total, 2 up, 3 in 2024-09-17T08:29:52.683 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:52 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:29:52.683 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:52 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:29:52.683 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:52 smithi118 ceph-mon[29491]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:52.684 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:52 smithi118 ceph-mon[29491]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2024-09-17T08:29:52.684 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:52 smithi118 ceph-mon[29491]: osdmap e17: 3 total, 2 up, 3 in 2024-09-17T08:29:52.684 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:52 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:29:52.684 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:52 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:29:52.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:52 smithi170 ceph-mon[33271]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:52.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:52 smithi170 ceph-mon[33271]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2024-09-17T08:29:52.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:52 smithi170 ceph-mon[33271]: osdmap e17: 3 total, 2 up, 3 in 2024-09-17T08:29:52.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:52 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:29:52.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:52 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:29:53.777 INFO:teuthology.orchestra.run.smithi118.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-17T08:29:53.825 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:53 smithi118 ceph-mon[29491]: purged_snaps scrub starts 2024-09-17T08:29:53.825 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:53 smithi118 ceph-mon[29491]: purged_snaps scrub ok 2024-09-17T08:29:53.825 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:53 smithi118 ceph-mon[29491]: osd.2 [v2:172.21.15.118:6818/1717873621,v1:172.21.15.118:6819/1717873621] boot 2024-09-17T08:29:53.825 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:53 smithi118 ceph-mon[29491]: osdmap e18: 3 total, 3 up, 3 in 2024-09-17T08:29:53.825 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:53 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:29:53.825 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:53 smithi118 ceph-mon[41418]: purged_snaps scrub starts 2024-09-17T08:29:53.826 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:53 smithi118 ceph-mon[41418]: purged_snaps scrub ok 2024-09-17T08:29:53.826 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:53 smithi118 ceph-mon[41418]: osd.2 [v2:172.21.15.118:6818/1717873621,v1:172.21.15.118:6819/1717873621] boot 2024-09-17T08:29:53.826 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:53 smithi118 ceph-mon[41418]: osdmap e18: 3 total, 3 up, 3 in 2024-09-17T08:29:53.826 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:53 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:29:53.851 INFO:teuthology.orchestra.run.smithi118.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-17T08:29:53.863 INFO:teuthology.orchestra.run.smithi118.stderr: stderr: 10+0 records in 2024-09-17T08:29:53.863 INFO:teuthology.orchestra.run.smithi118.stderr:10+0 records out 2024-09-17T08:29:53.863 INFO:teuthology.orchestra.run.smithi118.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00978655 s, 1.1 GB/s 2024-09-17T08:29:53.863 INFO:teuthology.orchestra.run.smithi118.stderr:--> Zapping successful for: 2024-09-17T08:29:53.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:53 smithi170 ceph-mon[33271]: purged_snaps scrub starts 2024-09-17T08:29:53.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:53 smithi170 ceph-mon[33271]: purged_snaps scrub ok 2024-09-17T08:29:53.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:53 smithi170 ceph-mon[33271]: osd.2 [v2:172.21.15.118:6818/1717873621,v1:172.21.15.118:6819/1717873621] boot 2024-09-17T08:29:53.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:53 smithi170 ceph-mon[33271]: osdmap e18: 3 total, 3 up, 3 in 2024-09-17T08:29:53.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:53 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:29:54.822 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:54 smithi118 ceph-mon[29491]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:54.822 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:54 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:29:54.822 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:54 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-17T08:29:54.822 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:54 smithi118 ceph-mon[29491]: osdmap e19: 3 total, 3 up, 3 in 2024-09-17T08:29:54.822 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:54 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:29:54.822 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:54 smithi118 ceph-mon[41418]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:54.822 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:54 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:29:54.822 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:54 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-17T08:29:54.822 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:54 smithi118 ceph-mon[41418]: osdmap e19: 3 total, 3 up, 3 in 2024-09-17T08:29:54.823 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:54 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:29:54.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:54 smithi170 ceph-mon[33271]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:29:54.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:54 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:29:54.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:54 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-17T08:29:54.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:54 smithi170 ceph-mon[33271]: osdmap e19: 3 total, 3 up, 3 in 2024-09-17T08:29:54.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:54 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:29:55.355 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi118:vg_nvme/lv_1 2024-09-17T08:29:55.643 INFO:journalctl@ceph.osd.0.smithi118.stdout:Sep 17 08:29:55 smithi118 sudo[62085]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-17T08:29:55.643 INFO:journalctl@ceph.osd.0.smithi118.stdout:Sep 17 08:29:55 smithi118 sudo[62085]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T08:29:55.644 INFO:journalctl@ceph.osd.0.smithi118.stdout:Sep 17 08:29:55 smithi118 sudo[62085]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T08:29:55.644 INFO:journalctl@ceph.osd.0.smithi118.stdout:Sep 17 08:29:55 smithi118 sudo[62085]: pam_unix(sudo:session): session closed for user root 2024-09-17T08:29:55.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:55 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-17T08:29:55.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:55 smithi170 ceph-mon[33271]: osdmap e20: 3 total, 3 up, 3 in 2024-09-17T08:29:55.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:55 smithi170 ceph-mon[33271]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:29:55.896 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:55 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-17T08:29:55.896 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:55 smithi118 ceph-mon[29491]: osdmap e20: 3 total, 3 up, 3 in 2024-09-17T08:29:55.896 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:55 smithi118 ceph-mon[29491]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:29:55.896 INFO:journalctl@ceph.osd.0.smithi118.stdout:Sep 17 08:29:55 smithi118 sudo[62088]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-17T08:29:55.896 INFO:journalctl@ceph.osd.0.smithi118.stdout:Sep 17 08:29:55 smithi118 sudo[62088]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T08:29:55.896 INFO:journalctl@ceph.osd.0.smithi118.stdout:Sep 17 08:29:55 smithi118 sudo[62088]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T08:29:55.896 INFO:journalctl@ceph.osd.0.smithi118.stdout:Sep 17 08:29:55 smithi118 sudo[62088]: pam_unix(sudo:session): session closed for user root 2024-09-17T08:29:55.896 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:55 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-17T08:29:55.897 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:55 smithi118 ceph-mon[41418]: osdmap e20: 3 total, 3 up, 3 in 2024-09-17T08:29:55.897 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:55 smithi118 ceph-mon[41418]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:29:56.271 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 17 08:29:56 smithi118 sudo[62098]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-17T08:29:56.271 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 17 08:29:56 smithi118 sudo[62098]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T08:29:56.271 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 17 08:29:56 smithi118 sudo[62098]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T08:29:56.271 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 17 08:29:56 smithi118 sudo[62098]: pam_unix(sudo:session): session closed for user root 2024-09-17T08:29:56.388 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:29:56.527 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 17 08:29:56 smithi118 sudo[62102]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-17T08:29:56.527 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 17 08:29:56 smithi118 sudo[62102]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T08:29:56.527 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 17 08:29:56 smithi118 sudo[62102]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T08:29:56.527 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 17 08:29:56 smithi118 sudo[62102]: pam_unix(sudo:session): session closed for user root 2024-09-17T08:29:56.831 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 17 08:29:56 smithi118 sudo[62211]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-17T08:29:56.831 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 17 08:29:56 smithi118 sudo[62211]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T08:29:56.831 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 17 08:29:56 smithi118 sudo[62211]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T08:29:56.831 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 17 08:29:56 smithi118 sudo[62211]: pam_unix(sudo:session): session closed for user root 2024-09-17T08:29:56.831 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:56 smithi118 ceph-mon[29491]: osdmap e21: 3 total, 3 up, 3 in 2024-09-17T08:29:56.831 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:56 smithi118 ceph-mon[29491]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-17T08:29:56.831 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:56 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:56.831 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:56 smithi118 ceph-mon[29491]: Detected new or changed devices on smithi118 2024-09-17T08:29:56.832 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:56 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:56.832 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:56 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:56.832 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:56 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:29:56.832 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:56 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:29:56.832 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:56 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:29:56.832 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:56 smithi118 ceph-mon[29491]: Adjusting osd_memory_target on smithi118 to 5354M 2024-09-17T08:29:56.832 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:56 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:56.832 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:56 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:29:56.832 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:56 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:29:56.832 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:56 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:56.832 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:56 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:29:56.832 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:56 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:29:56.832 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:56 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:29:56.832 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:56 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:56.833 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:56 smithi118 ceph-mon[41418]: osdmap e21: 3 total, 3 up, 3 in 2024-09-17T08:29:56.833 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:56 smithi118 ceph-mon[41418]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-17T08:29:56.833 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:56 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:56.833 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:56 smithi118 ceph-mon[41418]: Detected new or changed devices on smithi118 2024-09-17T08:29:56.833 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:56 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:56.833 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:56 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:56.833 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:56 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:29:56.833 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:56 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:29:56.833 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:56 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:29:56.833 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:56 smithi118 ceph-mon[41418]: Adjusting osd_memory_target on smithi118 to 5354M 2024-09-17T08:29:56.833 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:56 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:56.833 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:56 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:29:56.834 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:56 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:29:56.834 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:56 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:56.834 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:56 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:29:56.834 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:56 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:29:56.834 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:56 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:29:56.834 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:56 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:56.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:56 smithi170 ceph-mon[33271]: osdmap e21: 3 total, 3 up, 3 in 2024-09-17T08:29:56.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:56 smithi170 ceph-mon[33271]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-17T08:29:56.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:56 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:56.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:56 smithi170 ceph-mon[33271]: Detected new or changed devices on smithi118 2024-09-17T08:29:56.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:56 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:56.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:56 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:56.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:56 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:29:56.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:56 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:29:56.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:56 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:29:56.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:56 smithi170 ceph-mon[33271]: Adjusting osd_memory_target on smithi118 to 5354M 2024-09-17T08:29:56.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:56 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:56.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:56 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:29:56.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:56 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:29:56.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:56 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:56.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:56 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:29:56.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:56 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:29:56.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:56 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:29:56.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:56 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:57.093 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:57 smithi118 sudo[62255]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-17T08:29:57.093 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:57 smithi118 sudo[62255]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T08:29:57.093 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:57 smithi118 sudo[62255]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T08:29:57.093 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:57 smithi118 sudo[62255]: pam_unix(sudo:session): session closed for user root 2024-09-17T08:29:57.094 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 17 08:29:56 smithi118 sudo[62252]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-17T08:29:57.094 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 17 08:29:56 smithi118 sudo[62252]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T08:29:57.094 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 17 08:29:56 smithi118 sudo[62252]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T08:29:57.094 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 17 08:29:56 smithi118 sudo[62252]: pam_unix(sudo:session): session closed for user root 2024-09-17T08:29:57.495 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:57 smithi118 sudo[62266]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-17T08:29:57.495 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:57 smithi118 sudo[62266]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T08:29:57.495 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:57 smithi118 sudo[62266]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T08:29:57.495 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:57 smithi118 sudo[62266]: pam_unix(sudo:session): session closed for user root 2024-09-17T08:29:57.841 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:57 smithi170 sudo[36023]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-17T08:29:57.841 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:57 smithi170 sudo[36023]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T08:29:57.841 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:57 smithi170 sudo[36023]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T08:29:57.841 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:57 smithi170 sudo[36023]: pam_unix(sudo:session): session closed for user root 2024-09-17T08:29:58.110 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:57 smithi118 ceph-mon[29491]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T08:29:58.110 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:57 smithi118 ceph-mon[29491]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T08:29:58.110 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:57 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:29:58.110 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:57 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:29:58.110 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:57 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:29:58.110 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:57 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:29:58.110 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:57 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:29:58.110 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:57 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:29:58.111 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:57 smithi118 ceph-mon[29491]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:29:58.111 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:57 smithi118 ceph-mon[41418]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T08:29:58.111 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:57 smithi118 ceph-mon[41418]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T08:29:58.111 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:57 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:29:58.111 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:57 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:29:58.111 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:57 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:29:58.111 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:57 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:29:58.111 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:57 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:29:58.111 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:57 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:29:58.111 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:57 smithi118 ceph-mon[41418]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:29:58.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:57 smithi170 sudo[36026]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-17T08:29:58.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:57 smithi170 sudo[36026]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T08:29:58.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:57 smithi170 sudo[36026]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T08:29:58.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:57 smithi170 sudo[36026]: pam_unix(sudo:session): session closed for user root 2024-09-17T08:29:58.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:57 smithi170 ceph-mon[33271]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T08:29:58.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:57 smithi170 ceph-mon[33271]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T08:29:58.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:57 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:29:58.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:57 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:29:58.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:57 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:29:58.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:57 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:29:58.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:57 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:29:58.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:57 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:29:58.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:57 smithi170 ceph-mon[33271]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:29:58.363 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:58 smithi118 sudo[62332]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-17T08:29:58.363 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:58 smithi118 sudo[62332]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T08:29:58.363 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:58 smithi118 sudo[62332]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T08:29:58.363 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:58 smithi118 sudo[62332]: pam_unix(sudo:session): session closed for user root 2024-09-17T08:29:58.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:58 smithi118 sudo[62335]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-17T08:29:58.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:58 smithi118 sudo[62335]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T08:29:58.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:58 smithi118 sudo[62335]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T08:29:58.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:58 smithi118 sudo[62335]: pam_unix(sudo:session): session closed for user root 2024-09-17T08:29:59.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:58 smithi170 ceph-mon[33271]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T08:29:59.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:58 smithi170 ceph-mon[33271]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T08:29:59.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:58 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:59.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:58 smithi170 ceph-mon[33271]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T08:29:59.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:58 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:29:59.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:58 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:29:59.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:58 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:29:59.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:58 smithi170 ceph-mon[33271]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T08:29:59.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:58 smithi170 ceph-mon[33271]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:29:59.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:58 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:29:59.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:58 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:29:59.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:58 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:29:59.158 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:58 smithi118 ceph-mon[29491]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T08:29:59.158 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:58 smithi118 ceph-mon[29491]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T08:29:59.158 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:58 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:59.158 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:58 smithi118 ceph-mon[29491]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T08:29:59.158 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:58 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:29:59.158 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:58 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:29:59.158 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:58 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:29:59.158 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:58 smithi118 ceph-mon[29491]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T08:29:59.159 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:58 smithi118 ceph-mon[29491]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:29:59.159 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:58 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:29:59.159 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:58 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:29:59.159 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:58 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:29:59.159 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:58 smithi118 ceph-mon[41418]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T08:29:59.159 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:58 smithi118 ceph-mon[41418]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T08:29:59.159 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:58 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:29:59.159 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:58 smithi118 ceph-mon[41418]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T08:29:59.159 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:58 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:29:59.159 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:58 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:29:59.159 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:58 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:29:59.159 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:58 smithi118 ceph-mon[41418]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T08:29:59.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:58 smithi118 ceph-mon[41418]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:29:59.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:58 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:29:59.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:58 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:29:59.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:58 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:29:59.914 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:29:59 smithi118 ceph-mon[29491]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:29:59.914 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:29:59 smithi118 ceph-mon[41418]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:30:00.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:29:59 smithi170 ceph-mon[33271]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:30:01.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:00 smithi118 ceph-mon[41418]: overall HEALTH_OK 2024-09-17T08:30:01.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:00 smithi118 ceph-mon[29491]: overall HEALTH_OK 2024-09-17T08:30:01.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:00 smithi170 ceph-mon[33271]: overall HEALTH_OK 2024-09-17T08:30:02.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:01 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/1617481293' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c7254e0-d2fd-4b11-a110-04f9d7aa04a7"}]: dispatch 2024-09-17T08:30:02.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:01 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/1617481293' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c7254e0-d2fd-4b11-a110-04f9d7aa04a7"}]': finished 2024-09-17T08:30:02.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:01 smithi170 ceph-mon[33271]: osdmap e22: 4 total, 3 up, 4 in 2024-09-17T08:30:02.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:01 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:30:02.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:01 smithi170 ceph-mon[33271]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:30:02.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:01 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/554128736' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:30:02.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:01 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/1617481293' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c7254e0-d2fd-4b11-a110-04f9d7aa04a7"}]: dispatch 2024-09-17T08:30:02.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:01 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/1617481293' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c7254e0-d2fd-4b11-a110-04f9d7aa04a7"}]': finished 2024-09-17T08:30:02.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:01 smithi118 ceph-mon[41418]: osdmap e22: 4 total, 3 up, 4 in 2024-09-17T08:30:02.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:01 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:30:02.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:01 smithi118 ceph-mon[41418]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:30:02.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:01 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/554128736' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:30:02.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:01 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/1617481293' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c7254e0-d2fd-4b11-a110-04f9d7aa04a7"}]: dispatch 2024-09-17T08:30:02.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:01 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/1617481293' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c7254e0-d2fd-4b11-a110-04f9d7aa04a7"}]': finished 2024-09-17T08:30:02.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:01 smithi118 ceph-mon[29491]: osdmap e22: 4 total, 3 up, 4 in 2024-09-17T08:30:02.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:01 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:30:02.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:01 smithi118 ceph-mon[29491]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:30:02.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:01 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/554128736' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:30:04.740 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:04 smithi118 ceph-mon[29491]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:30:04.740 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:04 smithi118 ceph-mon[41418]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:30:04.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:04 smithi170 ceph-mon[33271]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:30:06.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:06 smithi118 ceph-mon[29491]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:30:06.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:06 smithi118 ceph-mon[41418]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:30:06.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:06 smithi170 ceph-mon[33271]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:30:07.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:07 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-17T08:30:07.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:07 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:30:07.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:07 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-17T08:30:07.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:07 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:30:07.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:07 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-17T08:30:07.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:07 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:30:08.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:08 smithi118 ceph-mon[29491]: Deploying daemon osd.3 on smithi118 2024-09-17T08:30:08.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:08 smithi118 ceph-mon[29491]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:30:08.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:08 smithi118 ceph-mon[41418]: Deploying daemon osd.3 on smithi118 2024-09-17T08:30:08.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:08 smithi118 ceph-mon[41418]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:30:08.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:08 smithi170 ceph-mon[33271]: Deploying daemon osd.3 on smithi118 2024-09-17T08:30:08.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:08 smithi170 ceph-mon[33271]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:30:10.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:10 smithi118 ceph-mon[29491]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:30:10.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:10 smithi118 ceph-mon[41418]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:30:10.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:10 smithi170 ceph-mon[33271]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:30:12.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:12 smithi118 ceph-mon[29491]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:30:12.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:12 smithi118 ceph-mon[41418]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:30:12.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:12 smithi170 ceph-mon[33271]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:30:14.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:14 smithi118 ceph-mon[41418]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:30:14.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:14 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:30:14.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:14 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:14.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:14 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:14.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:14 smithi118 ceph-mon[29491]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:30:14.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:14 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:30:14.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:14 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:14.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:14 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:14.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:14 smithi170 ceph-mon[33271]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:30:14.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:14 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:30:14.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:14 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:14.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:14 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:16.618 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:16 smithi118 ceph-mon[29491]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:30:16.618 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:16 smithi118 ceph-mon[41418]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:30:16.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:16 smithi170 ceph-mon[33271]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:30:17.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:17 smithi118 ceph-mon[29491]: from='osd.3 [v2:172.21.15.118:6826/1195800036,v1:172.21.15.118:6827/1195800036]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-17T08:30:17.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:17 smithi118 ceph-mon[41418]: from='osd.3 [v2:172.21.15.118:6826/1195800036,v1:172.21.15.118:6827/1195800036]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-17T08:30:17.583 INFO:teuthology.orchestra.run.smithi118.stdout:Created osd(s) 3 on host 'smithi118' 2024-09-17T08:30:17.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:17 smithi170 ceph-mon[33271]: from='osd.3 [v2:172.21.15.118:6826/1195800036,v1:172.21.15.118:6827/1195800036]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-17T08:30:18.229 DEBUG:teuthology.orchestra.run.smithi118:osd.3> sudo journalctl -f -n 0 -u ceph-7862d192-74ce-11ef-bceb-c7b262605968@osd.3.service 2024-09-17T08:30:18.232 INFO:tasks.cephadm:Deploying osd.4 on smithi170 with /dev/vg_nvme/lv_4... 2024-09-17T08:30:18.232 DEBUG:teuthology.orchestra.run.smithi170:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-17T08:30:18.443 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:30:18.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:18 smithi118 ceph-mon[29491]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:30:18.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:18 smithi118 ceph-mon[29491]: from='osd.3 [v2:172.21.15.118:6826/1195800036,v1:172.21.15.118:6827/1195800036]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-17T08:30:18.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:18 smithi118 ceph-mon[29491]: osdmap e23: 4 total, 3 up, 4 in 2024-09-17T08:30:18.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:18 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:30:18.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:18 smithi118 ceph-mon[29491]: from='osd.3 [v2:172.21.15.118:6826/1195800036,v1:172.21.15.118:6827/1195800036]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-17T08:30:18.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:18 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:18.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:18 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:18.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:18 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:18.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:18 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:18.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:18 smithi118 ceph-mon[41418]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:30:18.529 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:18 smithi118 ceph-mon[41418]: from='osd.3 [v2:172.21.15.118:6826/1195800036,v1:172.21.15.118:6827/1195800036]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-17T08:30:18.529 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:18 smithi118 ceph-mon[41418]: osdmap e23: 4 total, 3 up, 4 in 2024-09-17T08:30:18.529 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:18 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:30:18.529 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:18 smithi118 ceph-mon[41418]: from='osd.3 [v2:172.21.15.118:6826/1195800036,v1:172.21.15.118:6827/1195800036]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-17T08:30:18.529 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:18 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:18.529 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:18 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:18.529 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:18 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:18.529 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:18 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:18.563 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:18 smithi170 ceph-mon[33271]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:30:18.563 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:18 smithi170 ceph-mon[33271]: from='osd.3 [v2:172.21.15.118:6826/1195800036,v1:172.21.15.118:6827/1195800036]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-17T08:30:18.563 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:18 smithi170 ceph-mon[33271]: osdmap e23: 4 total, 3 up, 4 in 2024-09-17T08:30:18.564 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:18 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:30:18.564 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:18 smithi170 ceph-mon[33271]: from='osd.3 [v2:172.21.15.118:6826/1195800036,v1:172.21.15.118:6827/1195800036]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-09-17T08:30:18.564 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:18 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:18.564 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:18 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:18.564 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:18 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:18.564 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:18 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:19.027 INFO:journalctl@ceph.osd.3.smithi118.stdout:Sep 17 08:30:18 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-3[64385]: 2024-09-17T08:30:18.685+0000 7fb51f536640 -1 osd.3 0 waiting for initial osdmap 2024-09-17T08:30:19.027 INFO:journalctl@ceph.osd.3.smithi118.stdout:Sep 17 08:30:18 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-3[64385]: 2024-09-17T08:30:18.689+0000 7fb517357640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T08:30:19.697 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:19 smithi170 ceph-mon[33271]: from='osd.3 [v2:172.21.15.118:6826/1195800036,v1:172.21.15.118:6827/1195800036]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2024-09-17T08:30:19.697 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:19 smithi170 ceph-mon[33271]: osdmap e24: 4 total, 3 up, 4 in 2024-09-17T08:30:19.697 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:19 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:30:19.698 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:19 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:30:19.698 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:19 smithi170 ceph-mon[33271]: from='osd.3 [v2:172.21.15.118:6826/1195800036,v1:172.21.15.118:6827/1195800036]' entity='osd.3' 2024-09-17T08:30:19.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:19 smithi118 ceph-mon[29491]: from='osd.3 [v2:172.21.15.118:6826/1195800036,v1:172.21.15.118:6827/1195800036]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2024-09-17T08:30:19.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:19 smithi118 ceph-mon[29491]: osdmap e24: 4 total, 3 up, 4 in 2024-09-17T08:30:19.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:19 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:30:19.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:19 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:30:19.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:19 smithi118 ceph-mon[29491]: from='osd.3 [v2:172.21.15.118:6826/1195800036,v1:172.21.15.118:6827/1195800036]' entity='osd.3' 2024-09-17T08:30:19.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:19 smithi118 ceph-mon[41418]: from='osd.3 [v2:172.21.15.118:6826/1195800036,v1:172.21.15.118:6827/1195800036]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2024-09-17T08:30:19.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:19 smithi118 ceph-mon[41418]: osdmap e24: 4 total, 3 up, 4 in 2024-09-17T08:30:19.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:19 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:30:19.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:19 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:30:19.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:19 smithi118 ceph-mon[41418]: from='osd.3 [v2:172.21.15.118:6826/1195800036,v1:172.21.15.118:6827/1195800036]' entity='osd.3' 2024-09-17T08:30:20.122 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-17T08:30:20.210 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-17T08:30:20.220 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2024-09-17T08:30:20.221 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2024-09-17T08:30:20.221 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00857265 s, 1.2 GB/s 2024-09-17T08:30:20.221 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2024-09-17T08:30:20.436 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:20 smithi118 ceph-mon[41418]: purged_snaps scrub starts 2024-09-17T08:30:20.436 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:20 smithi118 ceph-mon[41418]: purged_snaps scrub ok 2024-09-17T08:30:20.436 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:20 smithi118 ceph-mon[41418]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:30:20.436 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:20 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:30:20.436 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:20 smithi118 ceph-mon[41418]: osd.3 [v2:172.21.15.118:6826/1195800036,v1:172.21.15.118:6827/1195800036] boot 2024-09-17T08:30:20.436 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:20 smithi118 ceph-mon[41418]: osdmap e25: 4 total, 4 up, 4 in 2024-09-17T08:30:20.436 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:20 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:30:20.574 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:20 smithi170 ceph-mon[33271]: purged_snaps scrub starts 2024-09-17T08:30:20.574 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:20 smithi170 ceph-mon[33271]: purged_snaps scrub ok 2024-09-17T08:30:20.575 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:20 smithi170 ceph-mon[33271]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:30:20.575 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:20 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:30:20.575 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:20 smithi170 ceph-mon[33271]: osd.3 [v2:172.21.15.118:6826/1195800036,v1:172.21.15.118:6827/1195800036] boot 2024-09-17T08:30:20.575 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:20 smithi170 ceph-mon[33271]: osdmap e25: 4 total, 4 up, 4 in 2024-09-17T08:30:20.575 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:20 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:30:20.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:20 smithi118 ceph-mon[29491]: purged_snaps scrub starts 2024-09-17T08:30:20.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:20 smithi118 ceph-mon[29491]: purged_snaps scrub ok 2024-09-17T08:30:20.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:20 smithi118 ceph-mon[29491]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:30:20.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:20 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:30:20.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:20 smithi118 ceph-mon[29491]: osd.3 [v2:172.21.15.118:6826/1195800036,v1:172.21.15.118:6827/1195800036] boot 2024-09-17T08:30:20.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:20 smithi118 ceph-mon[29491]: osdmap e25: 4 total, 4 up, 4 in 2024-09-17T08:30:20.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:20 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:30:20.901 DEBUG:teuthology.orchestra.run.smithi170:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi170:vg_nvme/lv_4 2024-09-17T08:30:21.092 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:30:21.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:21 smithi118 ceph-mon[29491]: osdmap e26: 4 total, 4 up, 4 in 2024-09-17T08:30:21.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:21 smithi118 ceph-mon[29491]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-17T08:30:21.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:21 smithi118 ceph-mon[29491]: Detected new or changed devices on smithi118 2024-09-17T08:30:21.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:21 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:21.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:21 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:21.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:21 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:30:21.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:21 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:30:21.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:21 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:30:21.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:21 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:30:21.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:21 smithi118 ceph-mon[29491]: Adjusting osd_memory_target on smithi118 to 4016M 2024-09-17T08:30:21.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:21 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:21.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:21 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:30:21.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:21 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:30:21.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:21 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:21.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:21 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:30:21.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:21 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:30:21.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:21 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:30:21.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:21 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:21.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:21 smithi118 ceph-mon[41418]: osdmap e26: 4 total, 4 up, 4 in 2024-09-17T08:30:21.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:21 smithi118 ceph-mon[41418]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-17T08:30:21.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:21 smithi118 ceph-mon[41418]: Detected new or changed devices on smithi118 2024-09-17T08:30:21.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:21.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:21.780 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:30:21.780 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:30:21.780 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:30:21.780 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:30:21.780 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:21 smithi118 ceph-mon[41418]: Adjusting osd_memory_target on smithi118 to 4016M 2024-09-17T08:30:21.780 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:21.780 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:30:21.780 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:30:21.780 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:21.780 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:30:21.780 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:30:21.780 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:30:21.781 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:21 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:21.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:21 smithi170 ceph-mon[33271]: osdmap e26: 4 total, 4 up, 4 in 2024-09-17T08:30:21.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:21 smithi170 ceph-mon[33271]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-17T08:30:21.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:21 smithi170 ceph-mon[33271]: Detected new or changed devices on smithi118 2024-09-17T08:30:21.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:21 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:21.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:21 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:21.897 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:21 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:30:21.897 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:21 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:30:21.897 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:21 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:30:21.897 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:21 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:30:21.898 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:21 smithi170 ceph-mon[33271]: Adjusting osd_memory_target on smithi118 to 4016M 2024-09-17T08:30:21.898 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:21 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:21.898 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:21 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:30:21.898 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:21 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:30:21.898 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:21 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:21.898 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:21 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:30:21.898 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:21 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:30:21.899 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:21 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:30:21.899 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:21 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:23.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:22 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:30:23.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:22 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:30:23.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:22 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:30:23.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:22 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:30:23.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:22 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:30:23.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:22 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:30:23.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:22 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:30:23.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:22 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:30:23.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:22 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:30:24.123 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:23 smithi170 ceph-mon[33271]: from='client.24181 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:30:24.123 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:23 smithi170 ceph-mon[33271]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-17T08:30:24.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:23 smithi118 ceph-mon[29491]: from='client.24181 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:30:24.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:23 smithi118 ceph-mon[29491]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-17T08:30:24.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:23 smithi118 ceph-mon[41418]: from='client.24181 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:30:24.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:23 smithi118 ceph-mon[41418]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-17T08:30:25.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:25 smithi118 ceph-mon[41418]: from='client.? 172.21.15.170:0/2733555642' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46dcae68-591d-41d4-9829-ce25368caffa"}]: dispatch 2024-09-17T08:30:25.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:25 smithi118 ceph-mon[41418]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46dcae68-591d-41d4-9829-ce25368caffa"}]: dispatch 2024-09-17T08:30:25.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:25 smithi118 ceph-mon[41418]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46dcae68-591d-41d4-9829-ce25368caffa"}]': finished 2024-09-17T08:30:25.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:25 smithi118 ceph-mon[41418]: osdmap e27: 5 total, 4 up, 5 in 2024-09-17T08:30:25.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:25 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:30:25.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:25 smithi118 ceph-mon[29491]: from='client.? 172.21.15.170:0/2733555642' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46dcae68-591d-41d4-9829-ce25368caffa"}]: dispatch 2024-09-17T08:30:25.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:25 smithi118 ceph-mon[29491]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46dcae68-591d-41d4-9829-ce25368caffa"}]: dispatch 2024-09-17T08:30:25.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:25 smithi118 ceph-mon[29491]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46dcae68-591d-41d4-9829-ce25368caffa"}]': finished 2024-09-17T08:30:25.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:25 smithi118 ceph-mon[29491]: osdmap e27: 5 total, 4 up, 5 in 2024-09-17T08:30:25.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:25 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:30:25.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:25 smithi170 ceph-mon[33271]: from='client.? 172.21.15.170:0/2733555642' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46dcae68-591d-41d4-9829-ce25368caffa"}]: dispatch 2024-09-17T08:30:25.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:25 smithi170 ceph-mon[33271]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46dcae68-591d-41d4-9829-ce25368caffa"}]: dispatch 2024-09-17T08:30:25.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:25 smithi170 ceph-mon[33271]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46dcae68-591d-41d4-9829-ce25368caffa"}]': finished 2024-09-17T08:30:25.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:25 smithi170 ceph-mon[33271]: osdmap e27: 5 total, 4 up, 5 in 2024-09-17T08:30:25.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:25 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:30:26.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:26 smithi118 ceph-mon[29491]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-17T08:30:26.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:26 smithi118 ceph-mon[29491]: from='client.? 172.21.15.170:0/650973151' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:30:26.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:26 smithi118 ceph-mon[41418]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-17T08:30:26.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:26 smithi118 ceph-mon[41418]: from='client.? 172.21.15.170:0/650973151' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:30:26.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:26 smithi170 ceph-mon[33271]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-17T08:30:26.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:26 smithi170 ceph-mon[33271]: from='client.? 172.21.15.170:0/650973151' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:30:28.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:28 smithi170 ceph-mon[33271]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-17T08:30:28.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:28 smithi118 ceph-mon[29491]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-17T08:30:28.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:28 smithi118 ceph-mon[41418]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-17T08:30:30.728 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:30 smithi170 ceph-mon[33271]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-17T08:30:30.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:30 smithi118 ceph-mon[29491]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-17T08:30:30.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:30 smithi118 ceph-mon[41418]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-17T08:30:31.764 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:31 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-17T08:30:31.764 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:31 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:30:31.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:31 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-17T08:30:31.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:31 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:30:31.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:31 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-17T08:30:31.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:31 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:30:32.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:32 smithi170 ceph-mon[33271]: Deploying daemon osd.4 on smithi170 2024-09-17T08:30:32.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:32 smithi170 ceph-mon[33271]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-17T08:30:32.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:32 smithi118 ceph-mon[29491]: Deploying daemon osd.4 on smithi170 2024-09-17T08:30:32.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:32 smithi118 ceph-mon[29491]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-17T08:30:32.679 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:32 smithi118 ceph-mon[41418]: Deploying daemon osd.4 on smithi170 2024-09-17T08:30:32.679 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:32 smithi118 ceph-mon[41418]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-17T08:30:34.570 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:34 smithi170 ceph-mon[33271]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-17T08:30:34.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:34 smithi118 ceph-mon[29491]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-17T08:30:34.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:34 smithi118 ceph-mon[41418]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-17T08:30:36.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:36 smithi170 ceph-mon[33271]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-17T08:30:36.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:36 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:30:36.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:36 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:36.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:36 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:36.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:36 smithi118 ceph-mon[29491]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-17T08:30:36.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:36 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:30:36.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:36 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:36.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:36 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:36.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:36 smithi118 ceph-mon[41418]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-17T08:30:36.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:36 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:30:36.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:36 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:36.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:36 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:38.695 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:38 smithi170 ceph-mon[33271]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-17T08:30:38.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:38 smithi118 ceph-mon[29491]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-17T08:30:38.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:38 smithi118 ceph-mon[41418]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-17T08:30:39.326 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 4 on host 'smithi170' 2024-09-17T08:30:39.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:39 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:39.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:39 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:39.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:39 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:30:39.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:39 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:30:39.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:39 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:39.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:39 smithi170 ceph-mon[33271]: from='osd.4 [v2:172.21.15.170:6800/1673072938,v1:172.21.15.170:6801/1673072938]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-17T08:30:39.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:39 smithi170 ceph-mon[33271]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-17T08:30:39.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:39 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:30:39.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:39 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:39.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:39 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:39.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:39 smithi170 ceph-mon[33271]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-17T08:30:39.917 DEBUG:teuthology.orchestra.run.smithi170:osd.4> sudo journalctl -f -n 0 -u ceph-7862d192-74ce-11ef-bceb-c7b262605968@osd.4.service 2024-09-17T08:30:39.920 INFO:tasks.cephadm:Deploying osd.5 on smithi170 with /dev/vg_nvme/lv_3... 2024-09-17T08:30:39.920 DEBUG:teuthology.orchestra.run.smithi170:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-17T08:30:40.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:39 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:40.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:39 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:40.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:39 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:30:40.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:39 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:30:40.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:39 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:40.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:39 smithi118 ceph-mon[29491]: from='osd.4 [v2:172.21.15.170:6800/1673072938,v1:172.21.15.170:6801/1673072938]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-17T08:30:40.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:39 smithi118 ceph-mon[29491]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-17T08:30:40.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:39 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:30:40.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:39 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:40.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:39 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:40.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:39 smithi118 ceph-mon[29491]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-17T08:30:40.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:39 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:40.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:39 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:40.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:39 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:30:40.029 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:39 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:30:40.029 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:39 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:40.029 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:39 smithi118 ceph-mon[41418]: from='osd.4 [v2:172.21.15.170:6800/1673072938,v1:172.21.15.170:6801/1673072938]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-17T08:30:40.029 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:39 smithi118 ceph-mon[41418]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-17T08:30:40.029 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:39 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:30:40.029 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:39 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:40.029 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:39 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:40.029 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:39 smithi118 ceph-mon[41418]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-17T08:30:40.694 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:30:40.810 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:40 smithi170 ceph-mon[33271]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-17T08:30:40.810 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:40 smithi170 ceph-mon[33271]: osdmap e28: 5 total, 4 up, 5 in 2024-09-17T08:30:40.810 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:40 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:30:40.810 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:40 smithi170 ceph-mon[33271]: from='osd.4 [v2:172.21.15.170:6800/1673072938,v1:172.21.15.170:6801/1673072938]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-17T08:30:40.810 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:40 smithi170 ceph-mon[33271]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-17T08:30:41.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:40 smithi118 ceph-mon[29491]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-17T08:30:41.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:40 smithi118 ceph-mon[29491]: osdmap e28: 5 total, 4 up, 5 in 2024-09-17T08:30:41.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:40 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:30:41.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:40 smithi118 ceph-mon[29491]: from='osd.4 [v2:172.21.15.170:6800/1673072938,v1:172.21.15.170:6801/1673072938]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-17T08:30:41.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:40 smithi118 ceph-mon[29491]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-17T08:30:41.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:40 smithi118 ceph-mon[41418]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-17T08:30:41.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:40 smithi118 ceph-mon[41418]: osdmap e28: 5 total, 4 up, 5 in 2024-09-17T08:30:41.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:40 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:30:41.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:40 smithi118 ceph-mon[41418]: from='osd.4 [v2:172.21.15.170:6800/1673072938,v1:172.21.15.170:6801/1673072938]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-17T08:30:41.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:40 smithi118 ceph-mon[41418]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-17T08:30:41.145 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 17 08:30:40 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-4[38685]: 2024-09-17T08:30:40.879+0000 7f60f67a8640 -1 osd.4 0 waiting for initial osdmap 2024-09-17T08:30:41.145 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 17 08:30:40 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-4[38685]: 2024-09-17T08:30:40.884+0000 7f60eddb2640 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T08:30:41.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:41 smithi170 ceph-mon[33271]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-09-17T08:30:41.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:41 smithi170 ceph-mon[33271]: osdmap e29: 5 total, 4 up, 5 in 2024-09-17T08:30:41.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:41 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:30:41.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:41 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:30:41.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:41 smithi170 ceph-mon[33271]: from='osd.4 ' entity='osd.4' 2024-09-17T08:30:41.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:41 smithi170 ceph-mon[33271]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-17T08:30:42.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:41 smithi118 ceph-mon[29491]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-09-17T08:30:42.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:41 smithi118 ceph-mon[29491]: osdmap e29: 5 total, 4 up, 5 in 2024-09-17T08:30:42.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:41 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:30:42.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:41 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:30:42.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:41 smithi118 ceph-mon[29491]: from='osd.4 ' entity='osd.4' 2024-09-17T08:30:42.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:41 smithi118 ceph-mon[29491]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-17T08:30:42.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:41 smithi118 ceph-mon[41418]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-09-17T08:30:42.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:41 smithi118 ceph-mon[41418]: osdmap e29: 5 total, 4 up, 5 in 2024-09-17T08:30:42.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:41 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:30:42.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:41 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:30:42.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:41 smithi118 ceph-mon[41418]: from='osd.4 ' entity='osd.4' 2024-09-17T08:30:42.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:41 smithi118 ceph-mon[41418]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-17T08:30:42.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:42 smithi170 ceph-mon[33271]: purged_snaps scrub starts 2024-09-17T08:30:42.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:42 smithi170 ceph-mon[33271]: purged_snaps scrub ok 2024-09-17T08:30:42.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:42 smithi170 ceph-mon[33271]: osd.4 [v2:172.21.15.170:6800/1673072938,v1:172.21.15.170:6801/1673072938] boot 2024-09-17T08:30:42.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:42 smithi170 ceph-mon[33271]: osdmap e30: 5 total, 5 up, 5 in 2024-09-17T08:30:42.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:42 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:30:43.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:42 smithi118 ceph-mon[29491]: purged_snaps scrub starts 2024-09-17T08:30:43.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:42 smithi118 ceph-mon[29491]: purged_snaps scrub ok 2024-09-17T08:30:43.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:42 smithi118 ceph-mon[29491]: osd.4 [v2:172.21.15.170:6800/1673072938,v1:172.21.15.170:6801/1673072938] boot 2024-09-17T08:30:43.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:42 smithi118 ceph-mon[29491]: osdmap e30: 5 total, 5 up, 5 in 2024-09-17T08:30:43.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:42 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:30:43.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:42 smithi118 ceph-mon[41418]: purged_snaps scrub starts 2024-09-17T08:30:43.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:42 smithi118 ceph-mon[41418]: purged_snaps scrub ok 2024-09-17T08:30:43.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:42 smithi118 ceph-mon[41418]: osd.4 [v2:172.21.15.170:6800/1673072938,v1:172.21.15.170:6801/1673072938] boot 2024-09-17T08:30:43.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:42 smithi118 ceph-mon[41418]: osdmap e30: 5 total, 5 up, 5 in 2024-09-17T08:30:43.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:42 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:30:43.186 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-17T08:30:43.282 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-17T08:30:43.294 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2024-09-17T08:30:43.294 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2024-09-17T08:30:43.294 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00953595 s, 1.1 GB/s 2024-09-17T08:30:43.294 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2024-09-17T08:30:43.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:43 smithi170 ceph-mon[33271]: osdmap e31: 5 total, 5 up, 5 in 2024-09-17T08:30:43.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:43 smithi170 ceph-mon[33271]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:30:44.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:43 smithi118 ceph-mon[29491]: osdmap e31: 5 total, 5 up, 5 in 2024-09-17T08:30:44.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:43 smithi118 ceph-mon[29491]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:30:44.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:43 smithi118 ceph-mon[41418]: osdmap e31: 5 total, 5 up, 5 in 2024-09-17T08:30:44.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:43 smithi118 ceph-mon[41418]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:30:44.278 DEBUG:teuthology.orchestra.run.smithi170:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi170:vg_nvme/lv_3 2024-09-17T08:30:44.468 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:30:44.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:44 smithi118 ceph-mon[29491]: osdmap e32: 5 total, 5 up, 5 in 2024-09-17T08:30:44.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:44 smithi118 ceph-mon[29491]: Detected new or changed devices on smithi170 2024-09-17T08:30:44.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:44 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:44.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:44 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:44.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:44 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:30:44.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:44 smithi118 ceph-mon[29491]: Adjusting osd_memory_target on smithi170 to 17088M 2024-09-17T08:30:44.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:44 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:44.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:44 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:30:44.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:44 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:30:44.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:44 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:44.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:44 smithi118 ceph-mon[41418]: osdmap e32: 5 total, 5 up, 5 in 2024-09-17T08:30:44.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:44 smithi118 ceph-mon[41418]: Detected new or changed devices on smithi170 2024-09-17T08:30:44.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:44 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:44.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:44 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:44.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:44 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:30:44.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:44 smithi118 ceph-mon[41418]: Adjusting osd_memory_target on smithi170 to 17088M 2024-09-17T08:30:44.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:44 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:44.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:44 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:30:44.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:44 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:30:44.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:44 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:44.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:44 smithi170 ceph-mon[33271]: osdmap e32: 5 total, 5 up, 5 in 2024-09-17T08:30:44.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:44 smithi170 ceph-mon[33271]: Detected new or changed devices on smithi170 2024-09-17T08:30:44.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:44 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:44.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:44 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:44.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:44 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:30:44.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:44 smithi170 ceph-mon[33271]: Adjusting osd_memory_target on smithi170 to 17088M 2024-09-17T08:30:44.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:44 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:44.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:44 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:30:44.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:44 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:30:44.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:44 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:30:45.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:45 smithi170 ceph-mon[33271]: osdmap e33: 5 total, 5 up, 5 in 2024-09-17T08:30:45.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:45 smithi170 ceph-mon[33271]: pgmap v117: 1 pgs: 1 remapped+peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:30:46.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:45 smithi118 ceph-mon[29491]: osdmap e33: 5 total, 5 up, 5 in 2024-09-17T08:30:46.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:45 smithi118 ceph-mon[29491]: pgmap v117: 1 pgs: 1 remapped+peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:30:46.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:45 smithi118 ceph-mon[41418]: osdmap e33: 5 total, 5 up, 5 in 2024-09-17T08:30:46.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:45 smithi118 ceph-mon[41418]: pgmap v117: 1 pgs: 1 remapped+peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:30:46.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:46 smithi170 ceph-mon[33271]: osdmap e34: 5 total, 5 up, 5 in 2024-09-17T08:30:46.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:46 smithi170 ceph-mon[33271]: from='client.24208 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:30:46.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:46 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:30:46.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:46 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:30:46.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:46 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:30:47.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:46 smithi118 ceph-mon[29491]: osdmap e34: 5 total, 5 up, 5 in 2024-09-17T08:30:47.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:46 smithi118 ceph-mon[29491]: from='client.24208 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:30:47.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:46 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:30:47.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:46 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:30:47.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:46 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:30:47.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:46 smithi118 ceph-mon[41418]: osdmap e34: 5 total, 5 up, 5 in 2024-09-17T08:30:47.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:46 smithi118 ceph-mon[41418]: from='client.24208 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:30:47.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:46 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:30:47.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:46 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:30:47.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:46 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:30:47.773 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:47 smithi170 ceph-mon[33271]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:30:48.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:47 smithi118 ceph-mon[29491]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:30:48.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:47 smithi118 ceph-mon[41418]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:30:48.990 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:48 smithi170 ceph-mon[33271]: from='client.? 172.21.15.170:0/2979163007' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d58f1343-ad2a-4153-89f7-e7d691aa5282"}]: dispatch 2024-09-17T08:30:48.990 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:48 smithi170 ceph-mon[33271]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d58f1343-ad2a-4153-89f7-e7d691aa5282"}]: dispatch 2024-09-17T08:30:48.990 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:48 smithi170 ceph-mon[33271]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d58f1343-ad2a-4153-89f7-e7d691aa5282"}]': finished 2024-09-17T08:30:48.990 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:48 smithi170 ceph-mon[33271]: osdmap e35: 6 total, 5 up, 6 in 2024-09-17T08:30:48.991 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:48 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:30:49.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:48 smithi118 ceph-mon[29491]: from='client.? 172.21.15.170:0/2979163007' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d58f1343-ad2a-4153-89f7-e7d691aa5282"}]: dispatch 2024-09-17T08:30:49.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:48 smithi118 ceph-mon[29491]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d58f1343-ad2a-4153-89f7-e7d691aa5282"}]: dispatch 2024-09-17T08:30:49.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:48 smithi118 ceph-mon[29491]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d58f1343-ad2a-4153-89f7-e7d691aa5282"}]': finished 2024-09-17T08:30:49.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:48 smithi118 ceph-mon[29491]: osdmap e35: 6 total, 5 up, 6 in 2024-09-17T08:30:49.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:48 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:30:49.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:48 smithi118 ceph-mon[41418]: from='client.? 172.21.15.170:0/2979163007' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d58f1343-ad2a-4153-89f7-e7d691aa5282"}]: dispatch 2024-09-17T08:30:49.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:48 smithi118 ceph-mon[41418]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d58f1343-ad2a-4153-89f7-e7d691aa5282"}]: dispatch 2024-09-17T08:30:49.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:48 smithi118 ceph-mon[41418]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d58f1343-ad2a-4153-89f7-e7d691aa5282"}]': finished 2024-09-17T08:30:49.029 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:48 smithi118 ceph-mon[41418]: osdmap e35: 6 total, 5 up, 6 in 2024-09-17T08:30:49.029 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:48 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:30:50.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:49 smithi118 ceph-mon[29491]: from='client.? 172.21.15.170:0/501934054' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:30:50.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:49 smithi118 ceph-mon[29491]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-17T08:30:50.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:49 smithi118 ceph-mon[41418]: from='client.? 172.21.15.170:0/501934054' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:30:50.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:49 smithi118 ceph-mon[41418]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-17T08:30:50.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:49 smithi170 ceph-mon[33271]: from='client.? 172.21.15.170:0/501934054' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:30:50.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:49 smithi170 ceph-mon[33271]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-17T08:30:52.474 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:52 smithi170 ceph-mon[33271]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-17T08:30:52.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:52 smithi118 ceph-mon[29491]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-17T08:30:52.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:52 smithi118 ceph-mon[41418]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-17T08:30:54.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:54 smithi118 ceph-mon[29491]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-17T08:30:54.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:54 smithi118 ceph-mon[41418]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-17T08:30:54.807 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:54 smithi170 ceph-mon[33271]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-17T08:30:55.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:55 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-17T08:30:55.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:55 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:30:55.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:55 smithi118 ceph-mon[29491]: Deploying daemon osd.5 on smithi170 2024-09-17T08:30:55.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:55 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-17T08:30:55.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:55 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:30:55.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:55 smithi118 ceph-mon[41418]: Deploying daemon osd.5 on smithi170 2024-09-17T08:30:55.841 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:55 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-17T08:30:55.841 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:55 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:30:55.841 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:55 smithi170 ceph-mon[33271]: Deploying daemon osd.5 on smithi170 2024-09-17T08:30:56.730 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:56 smithi170 ceph-mon[33271]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-17T08:30:56.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:56 smithi118 ceph-mon[29491]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-17T08:30:56.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:56 smithi118 ceph-mon[41418]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-17T08:30:58.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:30:58 smithi170 ceph-mon[33271]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-17T08:30:58.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:30:58 smithi118 ceph-mon[29491]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-17T08:30:58.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:30:58 smithi118 ceph-mon[41418]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-17T08:31:00.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:00 smithi118 ceph-mon[29491]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-17T08:31:00.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:00 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:31:00.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:00 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:00.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:00 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:00.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:00 smithi118 ceph-mon[41418]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-17T08:31:00.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:00 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:31:00.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:00 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:00.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:00 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:00.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:00 smithi170 ceph-mon[33271]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-17T08:31:00.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:00 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:31:00.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:00 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:00.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:00 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:02.686 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:02 smithi170 ceph-mon[33271]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:31:02.686 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:02 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:02.687 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:02 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:02.687 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:02 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:31:02.687 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:02 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:31:02.687 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:02 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:02.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:02 smithi118 ceph-mon[29491]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:31:02.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:02 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:02.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:02 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:02.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:02 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:31:02.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:02 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:31:02.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:02 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:02.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:02 smithi118 ceph-mon[41418]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:31:02.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:02 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:02.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:02 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:02.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:02 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:31:02.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:02 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:31:02.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:02 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:02.840 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 5 on host 'smithi170' 2024-09-17T08:31:03.513 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:03 smithi170 ceph-mon[33271]: from='osd.5 [v2:172.21.15.170:6808/3749911609,v1:172.21.15.170:6809/3749911609]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-17T08:31:03.513 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:03 smithi170 ceph-mon[33271]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-17T08:31:03.513 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:03 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:31:03.513 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:03 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:03.513 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:03 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:03.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:03 smithi118 ceph-mon[29491]: from='osd.5 [v2:172.21.15.170:6808/3749911609,v1:172.21.15.170:6809/3749911609]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-17T08:31:03.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:03 smithi118 ceph-mon[29491]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-17T08:31:03.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:03 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:31:03.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:03 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:03.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:03 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:03.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:03 smithi118 ceph-mon[41418]: from='osd.5 [v2:172.21.15.170:6808/3749911609,v1:172.21.15.170:6809/3749911609]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-17T08:31:03.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:03 smithi118 ceph-mon[41418]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-17T08:31:03.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:03 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:31:03.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:03 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:03.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:03 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:03.535 DEBUG:teuthology.orchestra.run.smithi170:osd.5> sudo journalctl -f -n 0 -u ceph-7862d192-74ce-11ef-bceb-c7b262605968@osd.5.service 2024-09-17T08:31:03.537 INFO:tasks.cephadm:Deploying osd.6 on smithi170 with /dev/vg_nvme/lv_2... 2024-09-17T08:31:03.537 DEBUG:teuthology.orchestra.run.smithi170:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-17T08:31:04.240 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:31:04.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:04 smithi170 ceph-mon[33271]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:31:04.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:04 smithi170 ceph-mon[33271]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-17T08:31:04.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:04 smithi170 ceph-mon[33271]: osdmap e36: 6 total, 5 up, 6 in 2024-09-17T08:31:04.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:04 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:31:04.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:04 smithi170 ceph-mon[33271]: from='osd.5 [v2:172.21.15.170:6808/3749911609,v1:172.21.15.170:6809/3749911609]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-17T08:31:04.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:04 smithi170 ceph-mon[33271]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-17T08:31:04.646 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:31:04 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5[44539]: 2024-09-17T08:31:04.444+0000 7f21db8b2640 -1 osd.5 0 waiting for initial osdmap 2024-09-17T08:31:04.646 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:31:04 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5[44539]: 2024-09-17T08:31:04.450+0000 7f21d76db640 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T08:31:04.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:04 smithi118 ceph-mon[29491]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:31:04.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:04 smithi118 ceph-mon[29491]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-17T08:31:04.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:04 smithi118 ceph-mon[29491]: osdmap e36: 6 total, 5 up, 6 in 2024-09-17T08:31:04.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:04 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:31:04.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:04 smithi118 ceph-mon[29491]: from='osd.5 [v2:172.21.15.170:6808/3749911609,v1:172.21.15.170:6809/3749911609]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-17T08:31:04.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:04 smithi118 ceph-mon[29491]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-17T08:31:04.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:04 smithi118 ceph-mon[41418]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:31:04.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:04 smithi118 ceph-mon[41418]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-17T08:31:04.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:04 smithi118 ceph-mon[41418]: osdmap e36: 6 total, 5 up, 6 in 2024-09-17T08:31:04.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:04 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:31:04.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:04 smithi118 ceph-mon[41418]: from='osd.5 [v2:172.21.15.170:6808/3749911609,v1:172.21.15.170:6809/3749911609]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-17T08:31:04.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:04 smithi118 ceph-mon[41418]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-17T08:31:05.642 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:05 smithi170 ceph-mon[33271]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-09-17T08:31:05.642 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:05 smithi170 ceph-mon[33271]: osdmap e37: 6 total, 5 up, 6 in 2024-09-17T08:31:05.642 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:05 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:31:05.642 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:05 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:31:05.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:05 smithi118 ceph-mon[29491]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-09-17T08:31:05.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:05 smithi118 ceph-mon[29491]: osdmap e37: 6 total, 5 up, 6 in 2024-09-17T08:31:05.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:05 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:31:05.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:05 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:31:05.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:05 smithi118 ceph-mon[41418]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-09-17T08:31:05.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:05 smithi118 ceph-mon[41418]: osdmap e37: 6 total, 5 up, 6 in 2024-09-17T08:31:05.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:05 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:31:05.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:05 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:31:06.557 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:06 smithi170 ceph-mon[33271]: purged_snaps scrub starts 2024-09-17T08:31:06.557 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:06 smithi170 ceph-mon[33271]: purged_snaps scrub ok 2024-09-17T08:31:06.557 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:06 smithi170 ceph-mon[33271]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:31:06.557 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:06 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:31:06.558 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:06 smithi170 ceph-mon[33271]: osd.5 [v2:172.21.15.170:6808/3749911609,v1:172.21.15.170:6809/3749911609] boot 2024-09-17T08:31:06.558 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:06 smithi170 ceph-mon[33271]: osdmap e38: 6 total, 6 up, 6 in 2024-09-17T08:31:06.558 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:06 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:31:06.599 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-17T08:31:06.694 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-17T08:31:06.707 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2024-09-17T08:31:06.707 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2024-09-17T08:31:06.707 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110302 s, 951 MB/s 2024-09-17T08:31:06.708 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2024-09-17T08:31:06.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:06 smithi118 ceph-mon[29491]: purged_snaps scrub starts 2024-09-17T08:31:06.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:06 smithi118 ceph-mon[29491]: purged_snaps scrub ok 2024-09-17T08:31:06.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:06 smithi118 ceph-mon[29491]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:31:06.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:06 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:31:06.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:06 smithi118 ceph-mon[29491]: osd.5 [v2:172.21.15.170:6808/3749911609,v1:172.21.15.170:6809/3749911609] boot 2024-09-17T08:31:06.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:06 smithi118 ceph-mon[29491]: osdmap e38: 6 total, 6 up, 6 in 2024-09-17T08:31:06.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:06 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:31:06.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:06 smithi118 ceph-mon[41418]: purged_snaps scrub starts 2024-09-17T08:31:06.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:06 smithi118 ceph-mon[41418]: purged_snaps scrub ok 2024-09-17T08:31:06.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:06 smithi118 ceph-mon[41418]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:31:06.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:06 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:31:06.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:06 smithi118 ceph-mon[41418]: osd.5 [v2:172.21.15.170:6808/3749911609,v1:172.21.15.170:6809/3749911609] boot 2024-09-17T08:31:06.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:06 smithi118 ceph-mon[41418]: osdmap e38: 6 total, 6 up, 6 in 2024-09-17T08:31:06.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:06 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:31:07.503 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:07 smithi170 ceph-mon[33271]: osdmap e39: 6 total, 6 up, 6 in 2024-09-17T08:31:07.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:07 smithi118 ceph-mon[29491]: osdmap e39: 6 total, 6 up, 6 in 2024-09-17T08:31:07.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:07 smithi118 ceph-mon[41418]: osdmap e39: 6 total, 6 up, 6 in 2024-09-17T08:31:08.003 DEBUG:teuthology.orchestra.run.smithi170:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi170:vg_nvme/lv_2 2024-09-17T08:31:08.200 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:31:08.568 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:08 smithi170 ceph-mon[33271]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:31:08.568 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:08 smithi170 ceph-mon[33271]: osdmap e40: 6 total, 6 up, 6 in 2024-09-17T08:31:08.568 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:08 smithi170 ceph-mon[33271]: Detected new or changed devices on smithi170 2024-09-17T08:31:08.568 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:08 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:08.568 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:08 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:08.568 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:08 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:31:08.568 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:08 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:31:08.568 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:08 smithi170 ceph-mon[33271]: Adjusting osd_memory_target on smithi170 to 8544M 2024-09-17T08:31:08.568 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:08 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:08.568 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:08 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:31:08.569 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:08 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:31:08.569 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:08 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:08.569 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:08 smithi170 ceph-mon[33271]: osdmap e41: 6 total, 6 up, 6 in 2024-09-17T08:31:08.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:08 smithi118 ceph-mon[29491]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:31:08.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:08 smithi118 ceph-mon[29491]: osdmap e40: 6 total, 6 up, 6 in 2024-09-17T08:31:08.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:08 smithi118 ceph-mon[29491]: Detected new or changed devices on smithi170 2024-09-17T08:31:08.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:08 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:08.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:08 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:08.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:08 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:31:08.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:08 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:31:08.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:08 smithi118 ceph-mon[29491]: Adjusting osd_memory_target on smithi170 to 8544M 2024-09-17T08:31:08.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:08 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:08.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:08 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:31:08.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:08 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:31:08.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:08 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:08.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:08 smithi118 ceph-mon[29491]: osdmap e41: 6 total, 6 up, 6 in 2024-09-17T08:31:08.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:08 smithi118 ceph-mon[41418]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:31:08.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:08 smithi118 ceph-mon[41418]: osdmap e40: 6 total, 6 up, 6 in 2024-09-17T08:31:08.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:08 smithi118 ceph-mon[41418]: Detected new or changed devices on smithi170 2024-09-17T08:31:08.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:08 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:08.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:08 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:08.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:08 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:31:08.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:08 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:31:08.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:08 smithi118 ceph-mon[41418]: Adjusting osd_memory_target on smithi170 to 8544M 2024-09-17T08:31:08.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:08 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:08.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:08 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:31:08.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:08 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:31:08.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:08 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:08.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:08 smithi118 ceph-mon[41418]: osdmap e41: 6 total, 6 up, 6 in 2024-09-17T08:31:09.866 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:09 smithi170 ceph-mon[33271]: osdmap e42: 6 total, 6 up, 6 in 2024-09-17T08:31:09.867 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:09 smithi170 ceph-mon[33271]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:31:10.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:09 smithi118 ceph-mon[29491]: osdmap e42: 6 total, 6 up, 6 in 2024-09-17T08:31:10.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:09 smithi118 ceph-mon[29491]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:31:10.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:09 smithi118 ceph-mon[41418]: osdmap e42: 6 total, 6 up, 6 in 2024-09-17T08:31:10.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:09 smithi118 ceph-mon[41418]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:31:10.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:10 smithi170 ceph-mon[33271]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-17T08:31:10.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:10 smithi170 ceph-mon[33271]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:31:10.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:10 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:31:10.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:10 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:31:10.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:10 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:31:11.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:10 smithi118 ceph-mon[29491]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-17T08:31:11.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:10 smithi118 ceph-mon[29491]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:31:11.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:10 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:31:11.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:10 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:31:11.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:10 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:31:11.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:10 smithi118 ceph-mon[41418]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-17T08:31:11.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:10 smithi118 ceph-mon[41418]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:31:11.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:10 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:31:11.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:10 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:31:11.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:10 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:31:12.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:11 smithi170 ceph-mon[33271]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:31:12.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:11 smithi118 ceph-mon[29491]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:31:12.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:11 smithi118 ceph-mon[41418]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:31:13.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:12 smithi170 ceph-mon[33271]: from='client.? 172.21.15.170:0/2237245332' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b63464b-4cfc-4225-91a4-ca4fe84adeb8"}]: dispatch 2024-09-17T08:31:13.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:12 smithi170 ceph-mon[33271]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b63464b-4cfc-4225-91a4-ca4fe84adeb8"}]: dispatch 2024-09-17T08:31:13.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:12 smithi170 ceph-mon[33271]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b63464b-4cfc-4225-91a4-ca4fe84adeb8"}]': finished 2024-09-17T08:31:13.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:12 smithi170 ceph-mon[33271]: osdmap e43: 7 total, 6 up, 7 in 2024-09-17T08:31:13.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:12 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:31:13.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:12 smithi170 ceph-mon[33271]: from='client.? 172.21.15.170:0/3957051237' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:31:13.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:12 smithi118 ceph-mon[29491]: from='client.? 172.21.15.170:0/2237245332' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b63464b-4cfc-4225-91a4-ca4fe84adeb8"}]: dispatch 2024-09-17T08:31:13.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:12 smithi118 ceph-mon[29491]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b63464b-4cfc-4225-91a4-ca4fe84adeb8"}]: dispatch 2024-09-17T08:31:13.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:12 smithi118 ceph-mon[29491]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b63464b-4cfc-4225-91a4-ca4fe84adeb8"}]': finished 2024-09-17T08:31:13.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:12 smithi118 ceph-mon[29491]: osdmap e43: 7 total, 6 up, 7 in 2024-09-17T08:31:13.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:12 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:31:13.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:12 smithi118 ceph-mon[29491]: from='client.? 172.21.15.170:0/3957051237' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:31:13.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:12 smithi118 ceph-mon[41418]: from='client.? 172.21.15.170:0/2237245332' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b63464b-4cfc-4225-91a4-ca4fe84adeb8"}]: dispatch 2024-09-17T08:31:13.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:12 smithi118 ceph-mon[41418]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b63464b-4cfc-4225-91a4-ca4fe84adeb8"}]: dispatch 2024-09-17T08:31:13.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:12 smithi118 ceph-mon[41418]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b63464b-4cfc-4225-91a4-ca4fe84adeb8"}]': finished 2024-09-17T08:31:13.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:12 smithi118 ceph-mon[41418]: osdmap e43: 7 total, 6 up, 7 in 2024-09-17T08:31:13.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:12 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:31:13.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:12 smithi118 ceph-mon[41418]: from='client.? 172.21.15.170:0/3957051237' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:31:14.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:13 smithi170 ceph-mon[33271]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-17T08:31:14.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:13 smithi118 ceph-mon[29491]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-17T08:31:14.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:13 smithi118 ceph-mon[41418]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-17T08:31:15.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:14 smithi170 ceph-mon[33271]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-17T08:31:15.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:14 smithi170 ceph-mon[33271]: Cluster is now healthy 2024-09-17T08:31:15.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:14 smithi118 ceph-mon[29491]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-17T08:31:15.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:14 smithi118 ceph-mon[29491]: Cluster is now healthy 2024-09-17T08:31:15.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:14 smithi118 ceph-mon[41418]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-17T08:31:15.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:14 smithi118 ceph-mon[41418]: Cluster is now healthy 2024-09-17T08:31:16.049 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:15 smithi170 ceph-mon[33271]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-17T08:31:16.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:15 smithi118 ceph-mon[29491]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-17T08:31:16.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:15 smithi118 ceph-mon[41418]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-17T08:31:18.765 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:18 smithi170 ceph-mon[33271]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-17T08:31:18.765 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:18 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-17T08:31:18.765 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:18 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:31:18.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:18 smithi118 ceph-mon[29491]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-17T08:31:18.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:18 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-17T08:31:18.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:18 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:31:18.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:18 smithi118 ceph-mon[41418]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-17T08:31:18.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:18 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-17T08:31:18.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:18 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:31:19.518 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:19 smithi170 ceph-mon[33271]: Deploying daemon osd.6 on smithi170 2024-09-17T08:31:19.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:19 smithi118 ceph-mon[29491]: Deploying daemon osd.6 on smithi170 2024-09-17T08:31:19.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:19 smithi118 ceph-mon[41418]: Deploying daemon osd.6 on smithi170 2024-09-17T08:31:20.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:20 smithi118 ceph-mon[29491]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-17T08:31:20.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:20 smithi118 ceph-mon[41418]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-17T08:31:20.842 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:20 smithi170 ceph-mon[33271]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-17T08:31:22.725 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:22 smithi170 ceph-mon[33271]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-17T08:31:22.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:22 smithi118 ceph-mon[29491]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-17T08:31:22.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:22 smithi118 ceph-mon[41418]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-17T08:31:23.701 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:23 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:31:23.701 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:23 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:23.701 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:23 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:23.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:23 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:31:23.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:23 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:23.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:23 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:23.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:23 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:31:23.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:23 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:23.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:23 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:24.686 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:24 smithi170 ceph-mon[33271]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:31:24.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:24 smithi118 ceph-mon[29491]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:31:24.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:24 smithi118 ceph-mon[41418]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:31:26.630 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:26 smithi170 ceph-mon[33271]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:31:26.630 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:26 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:26.630 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:26 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:26.630 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:26 smithi170 ceph-mon[33271]: from='osd.6 [v2:172.21.15.170:6816/965836527,v1:172.21.15.170:6817/965836527]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-17T08:31:26.630 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:26 smithi170 ceph-mon[33271]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-17T08:31:26.661 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 6 on host 'smithi170' 2024-09-17T08:31:26.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:26 smithi118 ceph-mon[29491]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:31:26.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:26 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:26.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:26 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:26.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:26 smithi118 ceph-mon[29491]: from='osd.6 [v2:172.21.15.170:6816/965836527,v1:172.21.15.170:6817/965836527]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-17T08:31:26.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:26 smithi118 ceph-mon[29491]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-17T08:31:26.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:26 smithi118 ceph-mon[41418]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:31:26.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:26 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:26.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:26 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:26.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:26 smithi118 ceph-mon[41418]: from='osd.6 [v2:172.21.15.170:6816/965836527,v1:172.21.15.170:6817/965836527]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-17T08:31:26.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:26 smithi118 ceph-mon[41418]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-17T08:31:27.271 DEBUG:teuthology.orchestra.run.smithi170:osd.6> sudo journalctl -f -n 0 -u ceph-7862d192-74ce-11ef-bceb-c7b262605968@osd.6.service 2024-09-17T08:31:27.274 INFO:tasks.cephadm:Deploying osd.7 on smithi170 with /dev/vg_nvme/lv_1... 2024-09-17T08:31:27.274 DEBUG:teuthology.orchestra.run.smithi170:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-17T08:31:27.528 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:31:27.911 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:27 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:27.911 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:27 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:27.911 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:27 smithi170 ceph-mon[33271]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-17T08:31:27.911 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:27 smithi170 ceph-mon[33271]: osdmap e44: 7 total, 6 up, 7 in 2024-09-17T08:31:27.911 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:27 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:31:27.911 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:27 smithi170 ceph-mon[33271]: from='osd.6 [v2:172.21.15.170:6816/965836527,v1:172.21.15.170:6817/965836527]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-17T08:31:27.911 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:27 smithi170 ceph-mon[33271]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-17T08:31:27.911 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:27 smithi170 ceph-mon[33271]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:31:28.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:27 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:28.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:27 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:28.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:27 smithi118 ceph-mon[29491]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-17T08:31:28.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:27 smithi118 ceph-mon[29491]: osdmap e44: 7 total, 6 up, 7 in 2024-09-17T08:31:28.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:27 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:31:28.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:27 smithi118 ceph-mon[29491]: from='osd.6 [v2:172.21.15.170:6816/965836527,v1:172.21.15.170:6817/965836527]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-17T08:31:28.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:27 smithi118 ceph-mon[29491]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-17T08:31:28.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:27 smithi118 ceph-mon[29491]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:31:28.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:27 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:28.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:27 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:28.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:27 smithi118 ceph-mon[41418]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-17T08:31:28.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:27 smithi118 ceph-mon[41418]: osdmap e44: 7 total, 6 up, 7 in 2024-09-17T08:31:28.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:27 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:31:28.029 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:27 smithi118 ceph-mon[41418]: from='osd.6 [v2:172.21.15.170:6816/965836527,v1:172.21.15.170:6817/965836527]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-17T08:31:28.029 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:27 smithi118 ceph-mon[41418]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-17T08:31:28.029 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:27 smithi118 ceph-mon[41418]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:31:28.354 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:31:27 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:31:27.922+0000 7fcaaa38a640 -1 osd.6 0 waiting for initial osdmap 2024-09-17T08:31:28.354 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:31:27 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:31:27.929+0000 7fcaa61b3640 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T08:31:29.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:28 smithi170 ceph-mon[33271]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-09-17T08:31:29.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:28 smithi170 ceph-mon[33271]: osdmap e45: 7 total, 6 up, 7 in 2024-09-17T08:31:29.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:28 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:31:29.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:28 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:31:29.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:28 smithi118 ceph-mon[29491]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-09-17T08:31:29.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:28 smithi118 ceph-mon[29491]: osdmap e45: 7 total, 6 up, 7 in 2024-09-17T08:31:29.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:28 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:31:29.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:28 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:31:29.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:28 smithi118 ceph-mon[41418]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-09-17T08:31:29.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:28 smithi118 ceph-mon[41418]: osdmap e45: 7 total, 6 up, 7 in 2024-09-17T08:31:29.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:28 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:31:29.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:28 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:31:30.018 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:29 smithi170 ceph-mon[33271]: purged_snaps scrub starts 2024-09-17T08:31:30.018 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:29 smithi170 ceph-mon[33271]: purged_snaps scrub ok 2024-09-17T08:31:30.018 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:29 smithi170 ceph-mon[33271]: osd.6 [v2:172.21.15.170:6816/965836527,v1:172.21.15.170:6817/965836527] boot 2024-09-17T08:31:30.018 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:29 smithi170 ceph-mon[33271]: osdmap e46: 7 total, 7 up, 7 in 2024-09-17T08:31:30.018 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:29 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:31:30.019 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:29 smithi170 ceph-mon[33271]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:31:30.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:29 smithi118 ceph-mon[29491]: purged_snaps scrub starts 2024-09-17T08:31:30.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:29 smithi118 ceph-mon[29491]: purged_snaps scrub ok 2024-09-17T08:31:30.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:29 smithi118 ceph-mon[29491]: osd.6 [v2:172.21.15.170:6816/965836527,v1:172.21.15.170:6817/965836527] boot 2024-09-17T08:31:30.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:29 smithi118 ceph-mon[29491]: osdmap e46: 7 total, 7 up, 7 in 2024-09-17T08:31:30.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:29 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:31:30.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:29 smithi118 ceph-mon[29491]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:31:30.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:29 smithi118 ceph-mon[41418]: purged_snaps scrub starts 2024-09-17T08:31:30.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:29 smithi118 ceph-mon[41418]: purged_snaps scrub ok 2024-09-17T08:31:30.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:29 smithi118 ceph-mon[41418]: osd.6 [v2:172.21.15.170:6816/965836527,v1:172.21.15.170:6817/965836527] boot 2024-09-17T08:31:30.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:29 smithi118 ceph-mon[41418]: osdmap e46: 7 total, 7 up, 7 in 2024-09-17T08:31:30.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:29 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:31:30.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:29 smithi118 ceph-mon[41418]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:31:30.283 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-17T08:31:30.283 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-17T08:31:30.283 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2024-09-17T08:31:30.283 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2024-09-17T08:31:30.283 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00968184 s, 1.1 GB/s 2024-09-17T08:31:30.283 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2024-09-17T08:31:30.874 DEBUG:teuthology.orchestra.run.smithi170:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi170:vg_nvme/lv_1 2024-09-17T08:31:31.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:30 smithi170 ceph-mon[33271]: osdmap e47: 7 total, 7 up, 7 in 2024-09-17T08:31:31.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:30 smithi118 ceph-mon[29491]: osdmap e47: 7 total, 7 up, 7 in 2024-09-17T08:31:31.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:30 smithi118 ceph-mon[41418]: osdmap e47: 7 total, 7 up, 7 in 2024-09-17T08:31:31.463 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:31:32.203 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:31 smithi170 ceph-mon[33271]: osdmap e48: 7 total, 7 up, 7 in 2024-09-17T08:31:32.203 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:31 smithi170 ceph-mon[33271]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:31:32.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:31 smithi118 ceph-mon[29491]: osdmap e48: 7 total, 7 up, 7 in 2024-09-17T08:31:32.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:31 smithi118 ceph-mon[29491]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:31:32.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:31 smithi118 ceph-mon[41418]: osdmap e48: 7 total, 7 up, 7 in 2024-09-17T08:31:32.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:31 smithi118 ceph-mon[41418]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:31:33.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:32 smithi170 ceph-mon[33271]: osdmap e49: 7 total, 7 up, 7 in 2024-09-17T08:31:33.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:32 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:33.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:32 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:33.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:32 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:31:33.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:32 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:31:33.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:32 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:31:33.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:32 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:33.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:32 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:31:33.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:32 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:31:33.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:32 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:33.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:32 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:31:33.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:32 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:31:33.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:32 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:31:33.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:32 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:33.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:32 smithi118 ceph-mon[29491]: osdmap e49: 7 total, 7 up, 7 in 2024-09-17T08:31:33.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:32 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:33.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:32 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:33.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:32 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:31:33.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:32 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:31:33.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:32 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:31:33.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:32 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:33.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:32 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:31:33.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:32 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:31:33.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:32 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:33.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:32 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:31:33.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:32 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:31:33.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:32 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:31:33.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:32 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:33.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:32 smithi118 ceph-mon[41418]: osdmap e49: 7 total, 7 up, 7 in 2024-09-17T08:31:33.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:32 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:33.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:32 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:33.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:32 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:31:33.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:32 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:31:33.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:32 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:31:33.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:32 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:33.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:32 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:31:33.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:32 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:31:33.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:32 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:33.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:32 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:31:33.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:32 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:31:33.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:32 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:31:33.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:32 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:34.252 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:33 smithi170 ceph-mon[33271]: Detected new or changed devices on smithi170 2024-09-17T08:31:34.252 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:33 smithi170 ceph-mon[33271]: Adjusting osd_memory_target on smithi170 to 5696M 2024-09-17T08:31:34.252 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:33 smithi170 ceph-mon[33271]: osdmap e50: 7 total, 7 up, 7 in 2024-09-17T08:31:34.252 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:33 smithi170 ceph-mon[33271]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:31:34.252 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:33 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:31:34.252 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:33 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:31:34.252 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:33 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:31:34.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:33 smithi118 ceph-mon[29491]: Detected new or changed devices on smithi170 2024-09-17T08:31:34.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:33 smithi118 ceph-mon[29491]: Adjusting osd_memory_target on smithi170 to 5696M 2024-09-17T08:31:34.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:33 smithi118 ceph-mon[29491]: osdmap e50: 7 total, 7 up, 7 in 2024-09-17T08:31:34.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:33 smithi118 ceph-mon[29491]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:31:34.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:33 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:31:34.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:33 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:31:34.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:33 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:31:34.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:33 smithi118 ceph-mon[41418]: Detected new or changed devices on smithi170 2024-09-17T08:31:34.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:33 smithi118 ceph-mon[41418]: Adjusting osd_memory_target on smithi170 to 5696M 2024-09-17T08:31:34.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:33 smithi118 ceph-mon[41418]: osdmap e50: 7 total, 7 up, 7 in 2024-09-17T08:31:34.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:33 smithi118 ceph-mon[41418]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:31:34.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:33 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:31:34.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:33 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:31:34.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:33 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:31:35.218 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:34 smithi170 ceph-mon[33271]: from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:31:35.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:34 smithi118 ceph-mon[29491]: from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:31:35.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:34 smithi118 ceph-mon[41418]: from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:31:36.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:35 smithi170 ceph-mon[33271]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-09-17T08:31:36.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:35 smithi118 ceph-mon[41418]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-09-17T08:31:36.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:35 smithi118 ceph-mon[29491]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-09-17T08:31:37.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:36 smithi170 ceph-mon[33271]: from='client.? 172.21.15.170:0/1909894271' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3431c0a5-527d-4199-b0e4-c17fc99f8981"}]: dispatch 2024-09-17T08:31:37.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:36 smithi170 ceph-mon[33271]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3431c0a5-527d-4199-b0e4-c17fc99f8981"}]: dispatch 2024-09-17T08:31:37.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:36 smithi170 ceph-mon[33271]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3431c0a5-527d-4199-b0e4-c17fc99f8981"}]': finished 2024-09-17T08:31:37.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:36 smithi170 ceph-mon[33271]: osdmap e51: 8 total, 7 up, 8 in 2024-09-17T08:31:37.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:36 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:31:37.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:36 smithi170 ceph-mon[33271]: from='client.? 172.21.15.170:0/2381976488' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:31:37.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:36 smithi118 ceph-mon[29491]: from='client.? 172.21.15.170:0/1909894271' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3431c0a5-527d-4199-b0e4-c17fc99f8981"}]: dispatch 2024-09-17T08:31:37.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:36 smithi118 ceph-mon[29491]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3431c0a5-527d-4199-b0e4-c17fc99f8981"}]: dispatch 2024-09-17T08:31:37.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:36 smithi118 ceph-mon[29491]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3431c0a5-527d-4199-b0e4-c17fc99f8981"}]': finished 2024-09-17T08:31:37.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:36 smithi118 ceph-mon[29491]: osdmap e51: 8 total, 7 up, 8 in 2024-09-17T08:31:37.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:36 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:31:37.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:36 smithi118 ceph-mon[29491]: from='client.? 172.21.15.170:0/2381976488' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:31:37.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:36 smithi118 ceph-mon[41418]: from='client.? 172.21.15.170:0/1909894271' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3431c0a5-527d-4199-b0e4-c17fc99f8981"}]: dispatch 2024-09-17T08:31:37.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:36 smithi118 ceph-mon[41418]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3431c0a5-527d-4199-b0e4-c17fc99f8981"}]: dispatch 2024-09-17T08:31:37.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:36 smithi118 ceph-mon[41418]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3431c0a5-527d-4199-b0e4-c17fc99f8981"}]': finished 2024-09-17T08:31:37.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:36 smithi118 ceph-mon[41418]: osdmap e51: 8 total, 7 up, 8 in 2024-09-17T08:31:37.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:36 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:31:37.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:36 smithi118 ceph-mon[41418]: from='client.? 172.21.15.170:0/2381976488' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:31:38.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:37 smithi118 ceph-mon[29491]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-17T08:31:38.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:37 smithi118 ceph-mon[41418]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-17T08:31:38.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:37 smithi170 ceph-mon[33271]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-17T08:31:40.642 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:40 smithi170 ceph-mon[33271]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-17T08:31:40.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:40 smithi118 ceph-mon[29491]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-17T08:31:40.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:40 smithi118 ceph-mon[41418]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-17T08:31:42.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:42 smithi170 ceph-mon[33271]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-17T08:31:42.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:42 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-17T08:31:42.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:42 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:31:42.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:42 smithi118 ceph-mon[29491]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-17T08:31:42.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:42 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-17T08:31:42.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:42 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:31:42.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:42 smithi118 ceph-mon[41418]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-17T08:31:42.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:42 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-17T08:31:42.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:42 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:31:43.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:43 smithi170 ceph-mon[33271]: Deploying daemon osd.7 on smithi170 2024-09-17T08:31:43.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:43 smithi118 ceph-mon[29491]: Deploying daemon osd.7 on smithi170 2024-09-17T08:31:43.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:43 smithi118 ceph-mon[41418]: Deploying daemon osd.7 on smithi170 2024-09-17T08:31:44.611 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:44 smithi170 ceph-mon[33271]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-17T08:31:44.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:44 smithi118 ceph-mon[29491]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-17T08:31:44.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:44 smithi118 ceph-mon[41418]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-17T08:31:46.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:46 smithi118 ceph-mon[29491]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:31:46.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:46 smithi118 ceph-mon[41418]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:31:46.854 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:46 smithi170 ceph-mon[33271]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:31:48.741 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:48 smithi170 ceph-mon[33271]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:31:48.741 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:48 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:31:48.741 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:48 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:48.741 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:48 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:48.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:48 smithi118 ceph-mon[29491]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:31:48.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:48 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:31:48.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:48 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:48.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:48 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:48.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:48 smithi118 ceph-mon[41418]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:31:48.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:48 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:31:48.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:48 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:48.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:48 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:50.527 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:50 smithi170 ceph-mon[33271]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:31:50.527 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:50 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:50.527 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:50 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:50.527 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:50 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:31:50.527 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:50 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:31:50.527 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:50 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:50.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:50 smithi118 ceph-mon[29491]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:31:50.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:50 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:50.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:50 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:50.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:50 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:31:50.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:50 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:31:50.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:50 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:50.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:50 smithi118 ceph-mon[41418]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:31:50.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:50 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:50.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:50 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:50.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:50 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:31:50.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:50 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:31:50.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:50 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:50.977 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 7 on host 'smithi170' 2024-09-17T08:31:51.612 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:51 smithi170 ceph-mon[33271]: from='osd.7 [v2:172.21.15.170:6824/3980860619,v1:172.21.15.170:6825/3980860619]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-17T08:31:51.612 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:51 smithi170 ceph-mon[33271]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-17T08:31:51.612 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:51 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:31:51.612 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:51 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:51.613 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:51 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:51.632 DEBUG:teuthology.orchestra.run.smithi170:osd.7> sudo journalctl -f -n 0 -u ceph-7862d192-74ce-11ef-bceb-c7b262605968@osd.7.service 2024-09-17T08:31:51.635 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-09-17T08:31:51.635 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-17T08:31:51.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:51 smithi118 ceph-mon[29491]: from='osd.7 [v2:172.21.15.170:6824/3980860619,v1:172.21.15.170:6825/3980860619]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-17T08:31:51.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:51 smithi118 ceph-mon[29491]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-17T08:31:51.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:51 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:31:51.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:51 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:51.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:51 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:51.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:51 smithi118 ceph-mon[41418]: from='osd.7 [v2:172.21.15.170:6824/3980860619,v1:172.21.15.170:6825/3980860619]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-17T08:31:51.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:51 smithi118 ceph-mon[41418]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-17T08:31:51.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:51 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:31:51.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:51 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:51.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:51 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:51.865 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:31:52.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:52 smithi118 ceph-mon[29491]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:31:52.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:52 smithi118 ceph-mon[29491]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-17T08:31:52.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:52 smithi118 ceph-mon[29491]: osdmap e52: 8 total, 7 up, 8 in 2024-09-17T08:31:52.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:52 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:31:52.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:52 smithi118 ceph-mon[29491]: from='osd.7 [v2:172.21.15.170:6824/3980860619,v1:172.21.15.170:6825/3980860619]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-17T08:31:52.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:52 smithi118 ceph-mon[29491]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-17T08:31:52.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:52 smithi118 ceph-mon[41418]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:31:52.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:52 smithi118 ceph-mon[41418]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-17T08:31:52.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:52 smithi118 ceph-mon[41418]: osdmap e52: 8 total, 7 up, 8 in 2024-09-17T08:31:52.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:52 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:31:52.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:52 smithi118 ceph-mon[41418]: from='osd.7 [v2:172.21.15.170:6824/3980860619,v1:172.21.15.170:6825/3980860619]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-17T08:31:52.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:52 smithi118 ceph-mon[41418]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-17T08:31:52.578 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:52 smithi170 ceph-mon[33271]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:31:52.578 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:52 smithi170 ceph-mon[33271]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-17T08:31:52.578 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:52 smithi170 ceph-mon[33271]: osdmap e52: 8 total, 7 up, 8 in 2024-09-17T08:31:52.578 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:52 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:31:52.578 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:52 smithi170 ceph-mon[33271]: from='osd.7 [v2:172.21.15.170:6824/3980860619,v1:172.21.15.170:6825/3980860619]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-17T08:31:52.578 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:52 smithi170 ceph-mon[33271]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-17T08:31:52.578 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:31:52 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:31:52.456+0000 7faf731ee640 -1 osd.7 0 waiting for initial osdmap 2024-09-17T08:31:52.579 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:31:52 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:31:52.463+0000 7faf6f017640 -1 osd.7 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T08:31:53.744 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:31:53.744 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:53 smithi118 ceph-mon[29491]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-09-17T08:31:53.744 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:53 smithi118 ceph-mon[29491]: osdmap e53: 8 total, 7 up, 8 in 2024-09-17T08:31:53.745 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:53 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:31:53.745 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:53 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:31:53.745 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:53 smithi118 ceph-mon[29491]: osd.7 [v2:172.21.15.170:6824/3980860619,v1:172.21.15.170:6825/3980860619] boot 2024-09-17T08:31:53.745 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:53 smithi118 ceph-mon[29491]: osdmap e54: 8 total, 8 up, 8 in 2024-09-17T08:31:53.745 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:53 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:31:53.745 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:53 smithi118 ceph-mon[41418]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-09-17T08:31:53.745 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:53 smithi118 ceph-mon[41418]: osdmap e53: 8 total, 7 up, 8 in 2024-09-17T08:31:53.745 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:53 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:31:53.745 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:53 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:31:53.745 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:53 smithi118 ceph-mon[41418]: osd.7 [v2:172.21.15.170:6824/3980860619,v1:172.21.15.170:6825/3980860619] boot 2024-09-17T08:31:53.745 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:53 smithi118 ceph-mon[41418]: osdmap e54: 8 total, 8 up, 8 in 2024-09-17T08:31:53.746 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:53 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:31:53.764 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:53 smithi170 ceph-mon[33271]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-09-17T08:31:53.764 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:53 smithi170 ceph-mon[33271]: osdmap e53: 8 total, 7 up, 8 in 2024-09-17T08:31:53.765 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:53 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:31:53.765 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:53 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:31:53.765 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:53 smithi170 ceph-mon[33271]: osd.7 [v2:172.21.15.170:6824/3980860619,v1:172.21.15.170:6825/3980860619] boot 2024-09-17T08:31:53.765 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:53 smithi170 ceph-mon[33271]: osdmap e54: 8 total, 8 up, 8 in 2024-09-17T08:31:53.765 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:53 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:31:54.555 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":54,"num_osds":8,"num_up_osds":8,"osd_up_since":1726561912,"num_in_osds":8,"osd_in_since":1726561896,"num_remapped_pgs":1} 2024-09-17T08:31:54.555 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-17T08:31:54.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:54 smithi170 ceph-mon[33271]: purged_snaps scrub starts 2024-09-17T08:31:54.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:54 smithi170 ceph-mon[33271]: purged_snaps scrub ok 2024-09-17T08:31:54.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:54 smithi170 ceph-mon[33271]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:31:54.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:54 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/310582349' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-17T08:31:54.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:54 smithi170 ceph-mon[33271]: osdmap e55: 8 total, 8 up, 8 in 2024-09-17T08:31:54.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:54 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:54.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:54 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:54.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:54 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:31:54.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:54 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:31:54.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:54 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:31:54.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:54 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:31:54.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:54 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:54.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:54 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:31:54.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:54 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:31:54.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:54 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:54.749 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:31:54.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:54 smithi118 ceph-mon[29491]: purged_snaps scrub starts 2024-09-17T08:31:54.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:54 smithi118 ceph-mon[29491]: purged_snaps scrub ok 2024-09-17T08:31:54.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:54 smithi118 ceph-mon[29491]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:31:54.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:54 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/310582349' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-17T08:31:54.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:54 smithi118 ceph-mon[29491]: osdmap e55: 8 total, 8 up, 8 in 2024-09-17T08:31:54.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:54 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:54.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:54 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:54.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:54 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:31:54.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:54 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:31:54.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:54 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:31:54.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:54 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:31:54.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:54 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:54.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:54 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:31:54.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:54 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:31:54.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:54 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:54.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:54 smithi118 ceph-mon[41418]: purged_snaps scrub starts 2024-09-17T08:31:54.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:54 smithi118 ceph-mon[41418]: purged_snaps scrub ok 2024-09-17T08:31:54.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:54 smithi118 ceph-mon[41418]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:31:54.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:54 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/310582349' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-17T08:31:54.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:54 smithi118 ceph-mon[41418]: osdmap e55: 8 total, 8 up, 8 in 2024-09-17T08:31:54.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:54 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:54.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:54 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:54.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:54 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:31:54.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:54 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:31:54.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:54 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:31:54.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:54 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:31:54.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:54 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:54.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:54 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:31:54.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:54 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:31:54.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:54 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:31:55.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:55 smithi118 ceph-mon[29491]: Detected new or changed devices on smithi170 2024-09-17T08:31:55.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:55 smithi118 ceph-mon[29491]: Adjusting osd_memory_target on smithi170 to 4272M 2024-09-17T08:31:55.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:55 smithi118 ceph-mon[29491]: osdmap e56: 8 total, 8 up, 8 in 2024-09-17T08:31:55.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:55 smithi118 ceph-mon[41418]: Detected new or changed devices on smithi170 2024-09-17T08:31:55.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:55 smithi118 ceph-mon[41418]: Adjusting osd_memory_target on smithi170 to 4272M 2024-09-17T08:31:55.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:55 smithi118 ceph-mon[41418]: osdmap e56: 8 total, 8 up, 8 in 2024-09-17T08:31:55.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:55 smithi170 ceph-mon[33271]: Detected new or changed devices on smithi170 2024-09-17T08:31:55.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:55 smithi170 ceph-mon[33271]: Adjusting osd_memory_target on smithi170 to 4272M 2024-09-17T08:31:55.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:55 smithi170 ceph-mon[33271]: osdmap e56: 8 total, 8 up, 8 in 2024-09-17T08:31:56.543 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:31:56.544 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":57,"fsid":"7862d192-74ce-11ef-bceb-c7b262605968","created":"2024-09-17T08:26:42.865266+0000","modified":"2024-09-17T08:31:55.820573+0000","last_up_change":"2024-09-17T08:31:52.813568+0000","last_in_change":"2024-09-17T08:31:36.166866+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T08:29:53.628517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ea41bec-a258-4d78-be6b-0299675f1106","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":3097740994},{"type":"v1","addr":"172.21.15.118:6803","nonce":3097740994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":3097740994},{"type":"v1","addr":"172.21.15.118:6805","nonce":3097740994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":3097740994},{"type":"v1","addr":"172.21.15.118:6809","nonce":3097740994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":3097740994},{"type":"v1","addr":"172.21.15.118:6807","nonce":3097740994}]},"public_addr":"172.21.15.118:6803/3097740994","cluster_addr":"172.21.15.118:6805/3097740994","heartbeat_back_addr":"172.21.15.118:6809/3097740994","heartbeat_front_addr":"172.21.15.118:6807/3097740994","state":["exists","up"]},{"osd":1,"uuid":"56073858-b245-4470-9941-1cdf67a69ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":3447428990},{"type":"v1","addr":"172.21.15.118:6811","nonce":3447428990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":3447428990},{"type":"v1","addr":"172.21.15.118:6813","nonce":3447428990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3447428990},{"type":"v1","addr":"172.21.15.118:6817","nonce":3447428990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":3447428990},{"type":"v1","addr":"172.21.15.118:6815","nonce":3447428990}]},"public_addr":"172.21.15.118:6811/3447428990","cluster_addr":"172.21.15.118:6813/3447428990","heartbeat_back_addr":"172.21.15.118:6817/3447428990","heartbeat_front_addr":"172.21.15.118:6815/3447428990","state":["exists","up"]},{"osd":2,"uuid":"edce62b8-f6c0-4c4a-b789-45304fa5eeb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1717873621},{"type":"v1","addr":"172.21.15.118:6819","nonce":1717873621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1717873621},{"type":"v1","addr":"172.21.15.118:6821","nonce":1717873621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1717873621},{"type":"v1","addr":"172.21.15.118:6825","nonce":1717873621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1717873621},{"type":"v1","addr":"172.21.15.118:6823","nonce":1717873621}]},"public_addr":"172.21.15.118:6819/1717873621","cluster_addr":"172.21.15.118:6821/1717873621","heartbeat_back_addr":"172.21.15.118:6825/1717873621","heartbeat_front_addr":"172.21.15.118:6823/1717873621","state":["exists","up"]},{"osd":3,"uuid":"1c7254e0-d2fd-4b11-a110-04f9d7aa04a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":1195800036},{"type":"v1","addr":"172.21.15.118:6827","nonce":1195800036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":1195800036},{"type":"v1","addr":"172.21.15.118:6829","nonce":1195800036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1195800036},{"type":"v1","addr":"172.21.15.118:6833","nonce":1195800036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":1195800036},{"type":"v1","addr":"172.21.15.118:6831","nonce":1195800036}]},"public_addr":"172.21.15.118:6827/1195800036","cluster_addr":"172.21.15.118:6829/1195800036","heartbeat_back_addr":"172.21.15.118:6833/1195800036","heartbeat_front_addr":"172.21.15.118:6831/1195800036","state":["exists","up"]},{"osd":4,"uuid":"46dcae68-591d-41d4-9829-ce25368caffa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1673072938},{"type":"v1","addr":"172.21.15.170:6801","nonce":1673072938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1673072938},{"type":"v1","addr":"172.21.15.170:6803","nonce":1673072938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1673072938},{"type":"v1","addr":"172.21.15.170:6807","nonce":1673072938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1673072938},{"type":"v1","addr":"172.21.15.170:6805","nonce":1673072938}]},"public_addr":"172.21.15.170:6801/1673072938","cluster_addr":"172.21.15.170:6803/1673072938","heartbeat_back_addr":"172.21.15.170:6807/1673072938","heartbeat_front_addr":"172.21.15.170:6805/1673072938","state":["exists","up"]},{"osd":5,"uuid":"d58f1343-ad2a-4153-89f7-e7d691aa5282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3749911609},{"type":"v1","addr":"172.21.15.170:6809","nonce":3749911609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3749911609},{"type":"v1","addr":"172.21.15.170:6811","nonce":3749911609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3749911609},{"type":"v1","addr":"172.21.15.170:6815","nonce":3749911609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3749911609},{"type":"v1","addr":"172.21.15.170:6813","nonce":3749911609}]},"public_addr":"172.21.15.170:6809/3749911609","cluster_addr":"172.21.15.170:6811/3749911609","heartbeat_back_addr":"172.21.15.170:6815/3749911609","heartbeat_front_addr":"172.21.15.170:6813/3749911609","state":["exists","up"]},{"osd":6,"uuid":"7b63464b-4cfc-4225-91a4-ca4fe84adeb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":965836527},{"type":"v1","addr":"172.21.15.170:6817","nonce":965836527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":965836527},{"type":"v1","addr":"172.21.15.170:6819","nonce":965836527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":965836527},{"type":"v1","addr":"172.21.15.170:6823","nonce":965836527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":965836527},{"type":"v1","addr":"172.21.15.170:6821","nonce":965836527}]},"public_addr":"172.21.15.170:6817/965836527","cluster_addr":"172.21.15.170:6819/965836527","heartbeat_back_addr":"172.21.15.170:6823/965836527","heartbeat_front_addr":"172.21.15.170:6821/965836527","state":["exists","up"]},{"osd":7,"uuid":"3431c0a5-527d-4199-b0e4-c17fc99f8981","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3980860619},{"type":"v1","addr":"172.21.15.170:6825","nonce":3980860619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3980860619},{"type":"v1","addr":"172.21.15.170:6827","nonce":3980860619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3980860619},{"type":"v1","addr":"172.21.15.170:6831","nonce":3980860619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3980860619},{"type":"v1","addr":"172.21.15.170:6829","nonce":3980860619}]},"public_addr":"172.21.15.170:6825/3980860619","cluster_addr":"172.21.15.170:6827/3980860619","heartbeat_back_addr":"172.21.15.170:6831/3980860619","heartbeat_front_addr":"172.21.15.170:6829/3980860619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:29:00.821543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:29:25.706259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:29:51.063464+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:30:17.959114+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:30:40.030347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:31:03.713849+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:31:27.429096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:31:51.600524+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.118:0/1798368975":"2024-09-18T08:27:37.329435+0000","172.21.15.118:0/2512398536":"2024-09-18T08:27:37.329435+0000","172.21.15.118:0/1111468500":"2024-09-18T08:27:37.329435+0000","172.21.15.118:6801/2223255332":"2024-09-18T08:27:37.329435+0000","172.21.15.118:6800/2223255332":"2024-09-18T08:27:37.329435+0000","172.21.15.118:6801/862659270":"2024-09-18T08:27:13.370329+0000","172.21.15.118:6800/862659270":"2024-09-18T08:27:13.370329+0000","172.21.15.118:0/858276273":"2024-09-18T08:27:13.370329+0000","172.21.15.118:0/3611715606":"2024-09-18T08:27:13.370329+0000","172.21.15.118:0/677224718":"2024-09-18T08:27:13.370329+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T08:31:56.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:56 smithi118 ceph-mon[29491]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:31:56.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:56 smithi118 ceph-mon[29491]: osdmap e57: 8 total, 8 up, 8 in 2024-09-17T08:31:56.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:56 smithi118 ceph-mon[41418]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:31:56.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:56 smithi118 ceph-mon[41418]: osdmap e57: 8 total, 8 up, 8 in 2024-09-17T08:31:56.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:56 smithi170 ceph-mon[33271]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:31:56.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:56 smithi170 ceph-mon[33271]: osdmap e57: 8 total, 8 up, 8 in 2024-09-17T08:31:57.281 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T08:29:53.628517+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T08:31:57.281 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-17T08:31:57.478 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:31:57.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:57 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3832071395' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T08:31:57.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:57 smithi118 ceph-mon[29491]: osdmap e58: 8 total, 8 up, 8 in 2024-09-17T08:31:57.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:57 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/3832071395' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T08:31:57.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:57 smithi118 ceph-mon[41418]: osdmap e58: 8 total, 8 up, 8 in 2024-09-17T08:31:57.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:57 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/3832071395' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T08:31:57.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:57 smithi170 ceph-mon[33271]: osdmap e58: 8 total, 8 up, 8 in 2024-09-17T08:31:58.774 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:58 smithi118 ceph-mon[29491]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:31:58.775 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:58 smithi118 ceph-mon[41418]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:31:58.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:58 smithi170 ceph-mon[33271]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:31:59.448 INFO:teuthology.orchestra.run.smithi118.stdout:pg_num: 1 2024-09-17T08:31:59.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:31:59 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/1524837551' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-17T08:31:59.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:31:59 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/1524837551' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-17T08:31:59.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:31:59 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/1524837551' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-17T08:32:00.154 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi118 2024-09-17T08:32:00.154 DEBUG:teuthology.orchestra.run.smithi170:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph orch apply rgw foo.a --placement '1;smithi118=foo.a' 2024-09-17T08:32:00.363 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:32:00.770 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:00 smithi170 ceph-mon[33271]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-09-17T08:32:00.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:00 smithi118 ceph-mon[29491]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-09-17T08:32:00.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:00 smithi118 ceph-mon[41418]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-09-17T08:32:02.157 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled rgw.foo.a update... 2024-09-17T08:32:02.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:02 smithi118 ceph-mon[41418]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-17T08:32:02.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:02 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:02.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:02 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:32:02.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:02 smithi118 ceph-mon[29491]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-17T08:32:02.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:02 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:02.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:02 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:32:02.888 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:02 smithi170 ceph-mon[33271]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-17T08:32:02.888 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:02 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:02.888 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:02 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:32:02.910 DEBUG:teuthology.orchestra.run.smithi118:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-7862d192-74ce-11ef-bceb-c7b262605968@rgw.foo.a.service 2024-09-17T08:32:02.913 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi170 2024-09-17T08:32:02.913 DEBUG:teuthology.orchestra.run.smithi170:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph osd pool create datapool 3 3 replicated 2024-09-17T08:32:03.108 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:32:03.500 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:03 smithi170 ceph-mon[33271]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi118=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:32:03.501 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:03 smithi170 ceph-mon[33271]: Saving service rgw.foo.a spec with placement smithi118=foo.a;count:1 2024-09-17T08:32:03.501 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:03 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:32:03.501 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:03 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:32:03.501 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:03 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:03.501 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:03 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-17T08:32:03.502 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:03 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-17T08:32:03.502 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:03 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:03.502 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:03 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:32:03.502 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:03 smithi170 ceph-mon[33271]: Deploying daemon rgw.foo.a on smithi118 2024-09-17T08:32:03.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:03 smithi118 ceph-mon[29491]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi118=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:32:03.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:03 smithi118 ceph-mon[29491]: Saving service rgw.foo.a spec with placement smithi118=foo.a;count:1 2024-09-17T08:32:03.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:03 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:32:03.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:03 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:32:03.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:03 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:03.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:03 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-17T08:32:03.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:03 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-17T08:32:03.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:03 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:03.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:03 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:32:03.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:03 smithi118 ceph-mon[29491]: Deploying daemon rgw.foo.a on smithi118 2024-09-17T08:32:03.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:03 smithi118 ceph-mon[41418]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi118=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:32:03.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:03 smithi118 ceph-mon[41418]: Saving service rgw.foo.a spec with placement smithi118=foo.a;count:1 2024-09-17T08:32:03.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:03 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:32:03.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:03 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:32:03.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:03 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:03.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:03 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-17T08:32:03.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:03 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-17T08:32:03.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:03 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:03.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:03 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:32:03.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:03 smithi118 ceph-mon[41418]: Deploying daemon rgw.foo.a on smithi118 2024-09-17T08:32:04.495 INFO:journalctl@ceph.rgw.foo.a.smithi118.stdout:Sep 17 08:32:04 smithi118 systemd[1]: Starting Ceph rgw.foo.a for 7862d192-74ce-11ef-bceb-c7b262605968... 2024-09-17T08:32:04.752 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:04 smithi118 ceph-mon[29491]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-17T08:32:04.752 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:04 smithi118 ceph-mon[41418]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-17T08:32:04.752 INFO:journalctl@ceph.rgw.foo.a.smithi118.stdout:Sep 17 08:32:04 smithi118 podman[69332]: 2024-09-17 08:32:04.493417503 +0000 UTC m=+0.015499127 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T08:32:04.753 INFO:journalctl@ceph.rgw.foo.a.smithi118.stdout:Sep 17 08:32:04 smithi118 podman[69332]: 2024-09-17 08:32:04.617155561 +0000 UTC m=+0.139237183 container create 48b9ec86736d0c21eccb98dfcaa4e1edcf6564f71ee13830c666bf3a410fc939 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-rgw-foo-a, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-17T08:32:04.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:04 smithi170 ceph-mon[33271]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-17T08:32:05.277 INFO:journalctl@ceph.rgw.foo.a.smithi118.stdout:Sep 17 08:32:04 smithi118 podman[69332]: 2024-09-17 08:32:04.870237988 +0000 UTC m=+0.392319616 container init 48b9ec86736d0c21eccb98dfcaa4e1edcf6564f71ee13830c666bf3a410fc939 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-rgw-foo-a, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-17T08:32:05.277 INFO:journalctl@ceph.rgw.foo.a.smithi118.stdout:Sep 17 08:32:04 smithi118 podman[69332]: 2024-09-17 08:32:04.874740287 +0000 UTC m=+0.396821909 container start 48b9ec86736d0c21eccb98dfcaa4e1edcf6564f71ee13830c666bf3a410fc939 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-rgw-foo-a, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-17T08:32:05.278 INFO:journalctl@ceph.rgw.foo.a.smithi118.stdout:Sep 17 08:32:04 smithi118 bash[69332]: 48b9ec86736d0c21eccb98dfcaa4e1edcf6564f71ee13830c666bf3a410fc939 2024-09-17T08:32:05.278 INFO:journalctl@ceph.rgw.foo.a.smithi118.stdout:Sep 17 08:32:04 smithi118 systemd[1]: Started Ceph rgw.foo.a for 7862d192-74ce-11ef-bceb-c7b262605968. 2024-09-17T08:32:05.504 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'datapool' created 2024-09-17T08:32:05.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:05 smithi118 ceph-mon[29491]: from='client.? 172.21.15.170:0/661427233' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-17T08:32:05.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:05 smithi118 ceph-mon[29491]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-17T08:32:05.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:05 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:05.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:05 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:05.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:05 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:05.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:05 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:05.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:05 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:05.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:05 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:32:05.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:05 smithi118 ceph-mon[41418]: from='client.? 172.21.15.170:0/661427233' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-17T08:32:05.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:05 smithi118 ceph-mon[41418]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-17T08:32:05.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:05 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:05.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:05 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:05.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:05 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:05.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:05 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:05.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:05 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:05.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:05 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:32:05.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:05 smithi170 ceph-mon[33271]: from='client.? 172.21.15.170:0/661427233' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-17T08:32:05.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:05 smithi170 ceph-mon[33271]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-17T08:32:05.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:05 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:05.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:05 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:05.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:05 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:05.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:05 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:05.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:05 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:05.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:05 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:32:06.238 DEBUG:teuthology.orchestra.run.smithi170:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- rbd pool init datapool 2024-09-17T08:32:06.436 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:32:06.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:06 smithi118 ceph-mon[29491]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-17T08:32:06.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:06 smithi118 ceph-mon[29491]: Saving service rgw.foo.a spec with placement smithi118=foo.a;count:1 2024-09-17T08:32:06.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:06 smithi118 ceph-mon[29491]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-17T08:32:06.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:06 smithi118 ceph-mon[29491]: osdmap e59: 8 total, 8 up, 8 in 2024-09-17T08:32:06.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:06 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/1196077201' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-17T08:32:06.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:06 smithi118 ceph-mon[29491]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-17T08:32:06.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:06 smithi118 ceph-mon[41418]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-17T08:32:06.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:06 smithi118 ceph-mon[41418]: Saving service rgw.foo.a spec with placement smithi118=foo.a;count:1 2024-09-17T08:32:06.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:06 smithi118 ceph-mon[41418]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-17T08:32:06.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:06 smithi118 ceph-mon[41418]: osdmap e59: 8 total, 8 up, 8 in 2024-09-17T08:32:06.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:06 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/1196077201' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-17T08:32:06.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:06 smithi118 ceph-mon[41418]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-17T08:32:06.813 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:06 smithi170 ceph-mon[33271]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-17T08:32:06.813 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:06 smithi170 ceph-mon[33271]: Saving service rgw.foo.a spec with placement smithi118=foo.a;count:1 2024-09-17T08:32:06.813 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:06 smithi170 ceph-mon[33271]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-17T08:32:06.814 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:06 smithi170 ceph-mon[33271]: osdmap e59: 8 total, 8 up, 8 in 2024-09-17T08:32:06.814 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:06 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/1196077201' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-17T08:32:06.814 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:06 smithi170 ceph-mon[33271]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-17T08:32:07.550 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:07 smithi170 ceph-mon[33271]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-17T08:32:07.551 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:07 smithi170 ceph-mon[33271]: osdmap e60: 8 total, 8 up, 8 in 2024-09-17T08:32:07.551 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:07 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:07.551 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:07 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:07.551 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:07 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:32:07.551 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:07 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:32:07.551 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:07 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:07.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:07 smithi118 ceph-mon[41418]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-17T08:32:07.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:07 smithi118 ceph-mon[41418]: osdmap e60: 8 total, 8 up, 8 in 2024-09-17T08:32:07.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:07 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:07.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:07 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:07.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:07 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:32:07.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:07 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:32:07.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:07 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:07.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:07 smithi118 ceph-mon[29491]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-17T08:32:07.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:07 smithi118 ceph-mon[29491]: osdmap e60: 8 total, 8 up, 8 in 2024-09-17T08:32:07.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:07 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:07.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:07 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:07.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:07 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:32:07.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:07 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:32:07.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:07 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:08.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:08 smithi118 ceph-mon[29491]: Checking dashboard <-> RGW credentials 2024-09-17T08:32:08.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:08 smithi118 ceph-mon[29491]: pgmap v185: 36 pgs: 6 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:32:08.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:08 smithi118 ceph-mon[29491]: osdmap e61: 8 total, 8 up, 8 in 2024-09-17T08:32:08.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:08 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/1196077201' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T08:32:08.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:08 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/4118988880' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T08:32:08.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:08 smithi118 ceph-mon[29491]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T08:32:08.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:08 smithi118 ceph-mon[29491]: from='client.? 172.21.15.170:0/1584632054' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-17T08:32:08.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:08 smithi118 ceph-mon[29491]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-17T08:32:08.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:08 smithi118 ceph-mon[29491]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T08:32:08.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:08 smithi118 ceph-mon[41418]: Checking dashboard <-> RGW credentials 2024-09-17T08:32:08.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:08 smithi118 ceph-mon[41418]: pgmap v185: 36 pgs: 6 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:32:08.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:08 smithi118 ceph-mon[41418]: osdmap e61: 8 total, 8 up, 8 in 2024-09-17T08:32:08.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:08 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/1196077201' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T08:32:08.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:08 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/4118988880' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T08:32:08.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:08 smithi118 ceph-mon[41418]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T08:32:08.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:08 smithi118 ceph-mon[41418]: from='client.? 172.21.15.170:0/1584632054' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-17T08:32:08.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:08 smithi118 ceph-mon[41418]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-17T08:32:08.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:08 smithi118 ceph-mon[41418]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T08:32:08.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:08 smithi170 ceph-mon[33271]: Checking dashboard <-> RGW credentials 2024-09-17T08:32:08.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:08 smithi170 ceph-mon[33271]: pgmap v185: 36 pgs: 6 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:32:08.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:08 smithi170 ceph-mon[33271]: osdmap e61: 8 total, 8 up, 8 in 2024-09-17T08:32:08.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:08 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/1196077201' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T08:32:08.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:08 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/4118988880' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T08:32:08.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:08 smithi170 ceph-mon[33271]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T08:32:08.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:08 smithi170 ceph-mon[33271]: from='client.? 172.21.15.170:0/1584632054' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-17T08:32:08.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:08 smithi170 ceph-mon[33271]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-17T08:32:08.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:08 smithi170 ceph-mon[33271]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T08:32:09.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:09 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/4118988880' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-17T08:32:09.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:09 smithi118 ceph-mon[29491]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-17T08:32:09.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:09 smithi118 ceph-mon[29491]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-17T08:32:09.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:09 smithi118 ceph-mon[29491]: osdmap e62: 8 total, 8 up, 8 in 2024-09-17T08:32:09.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:09 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/4118988880' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-17T08:32:09.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:09 smithi118 ceph-mon[41418]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-17T08:32:09.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:09 smithi118 ceph-mon[41418]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-17T08:32:09.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:09 smithi118 ceph-mon[41418]: osdmap e62: 8 total, 8 up, 8 in 2024-09-17T08:32:09.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:09 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/4118988880' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-17T08:32:09.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:09 smithi170 ceph-mon[33271]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-17T08:32:09.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:09 smithi170 ceph-mon[33271]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-17T08:32:09.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:09 smithi170 ceph-mon[33271]: osdmap e62: 8 total, 8 up, 8 in 2024-09-17T08:32:10.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:10 smithi118 ceph-mon[29491]: pgmap v188: 68 pgs: 20 creating+peering, 20 unknown, 28 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-09-17T08:32:10.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:10 smithi118 ceph-mon[29491]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-17T08:32:10.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:10 smithi118 ceph-mon[29491]: Cluster is now healthy 2024-09-17T08:32:10.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:10 smithi118 ceph-mon[29491]: osdmap e63: 8 total, 8 up, 8 in 2024-09-17T08:32:10.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:10 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/1196077201' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T08:32:10.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:10 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/4118988880' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T08:32:10.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:10 smithi118 ceph-mon[29491]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T08:32:10.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:10 smithi118 ceph-mon[41418]: pgmap v188: 68 pgs: 20 creating+peering, 20 unknown, 28 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-09-17T08:32:10.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:10 smithi118 ceph-mon[41418]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-17T08:32:10.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:10 smithi118 ceph-mon[41418]: Cluster is now healthy 2024-09-17T08:32:10.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:10 smithi118 ceph-mon[41418]: osdmap e63: 8 total, 8 up, 8 in 2024-09-17T08:32:10.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:10 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/1196077201' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T08:32:10.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:10 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/4118988880' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T08:32:10.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:10 smithi118 ceph-mon[41418]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T08:32:10.857 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:10 smithi170 ceph-mon[33271]: pgmap v188: 68 pgs: 20 creating+peering, 20 unknown, 28 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-09-17T08:32:10.857 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:10 smithi170 ceph-mon[33271]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-17T08:32:10.857 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:10 smithi170 ceph-mon[33271]: Cluster is now healthy 2024-09-17T08:32:10.857 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:10 smithi170 ceph-mon[33271]: osdmap e63: 8 total, 8 up, 8 in 2024-09-17T08:32:10.858 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:10 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/1196077201' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T08:32:10.858 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:10 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/4118988880' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T08:32:10.858 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:10 smithi170 ceph-mon[33271]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T08:32:11.158 DEBUG:teuthology.orchestra.run.smithi170:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.170 --placement '1;smithi170=iscsi.a' 2024-09-17T08:32:11.355 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:32:11.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:11 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/4118988880' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-17T08:32:11.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:11 smithi118 ceph-mon[29491]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-17T08:32:11.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:11 smithi118 ceph-mon[29491]: osdmap e64: 8 total, 8 up, 8 in 2024-09-17T08:32:11.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:11 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/4118988880' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-17T08:32:11.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:11 smithi118 ceph-mon[41418]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-17T08:32:11.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:11 smithi118 ceph-mon[41418]: osdmap e64: 8 total, 8 up, 8 in 2024-09-17T08:32:11.785 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:11 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/4118988880' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-17T08:32:11.786 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:11 smithi170 ceph-mon[33271]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-17T08:32:11.786 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:11 smithi170 ceph-mon[33271]: osdmap e64: 8 total, 8 up, 8 in 2024-09-17T08:32:12.797 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:12 smithi170 ceph-mon[33271]: pgmap v191: 100 pgs: 24 creating+peering, 35 unknown, 41 active+clean; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-17T08:32:12.797 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:12 smithi170 ceph-mon[33271]: osdmap e65: 8 total, 8 up, 8 in 2024-09-17T08:32:12.797 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:12 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/2275607284' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T08:32:12.797 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:12 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/2419221670' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T08:32:12.797 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:12 smithi170 ceph-mon[33271]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T08:32:12.797 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:12 smithi170 ceph-mon[33271]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T08:32:13.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:12 smithi118 ceph-mon[29491]: pgmap v191: 100 pgs: 24 creating+peering, 35 unknown, 41 active+clean; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-17T08:32:13.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:12 smithi118 ceph-mon[29491]: osdmap e65: 8 total, 8 up, 8 in 2024-09-17T08:32:13.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:12 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/2275607284' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T08:32:13.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:12 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/2419221670' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T08:32:13.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:12 smithi118 ceph-mon[29491]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T08:32:13.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:12 smithi118 ceph-mon[29491]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T08:32:13.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:12 smithi118 ceph-mon[41418]: pgmap v191: 100 pgs: 24 creating+peering, 35 unknown, 41 active+clean; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-17T08:32:13.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:12 smithi118 ceph-mon[41418]: osdmap e65: 8 total, 8 up, 8 in 2024-09-17T08:32:13.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:12 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/2275607284' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T08:32:13.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:12 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/2419221670' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T08:32:13.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:12 smithi118 ceph-mon[41418]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T08:32:13.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:12 smithi118 ceph-mon[41418]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T08:32:13.260 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled iscsi.datapool update... 2024-09-17T08:32:13.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:13 smithi118 ceph-mon[29491]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-17T08:32:13.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:13 smithi118 ceph-mon[29491]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-17T08:32:13.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:13 smithi118 ceph-mon[29491]: osdmap e66: 8 total, 8 up, 8 in 2024-09-17T08:32:13.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:13 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/2419221670' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T08:32:13.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:13 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/2275607284' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T08:32:13.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:13 smithi118 ceph-mon[29491]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T08:32:13.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:13 smithi118 ceph-mon[29491]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T08:32:13.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:13 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:13.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:13 smithi118 ceph-mon[41418]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-17T08:32:13.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:13 smithi118 ceph-mon[41418]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-17T08:32:13.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:13 smithi118 ceph-mon[41418]: osdmap e66: 8 total, 8 up, 8 in 2024-09-17T08:32:13.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:13 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/2419221670' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T08:32:13.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:13 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/2275607284' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T08:32:13.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:13 smithi118 ceph-mon[41418]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T08:32:13.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:13 smithi118 ceph-mon[41418]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T08:32:13.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:13 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:13.890 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:13 smithi170 ceph-mon[33271]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-17T08:32:13.890 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:13 smithi170 ceph-mon[33271]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-17T08:32:13.890 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:13 smithi170 ceph-mon[33271]: osdmap e66: 8 total, 8 up, 8 in 2024-09-17T08:32:13.890 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:13 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/2419221670' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T08:32:13.890 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:13 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/2275607284' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T08:32:13.891 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:13 smithi170 ceph-mon[33271]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T08:32:13.891 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:13 smithi170 ceph-mon[33271]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T08:32:13.891 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:13 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:13.912 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-09-17T08:32:13.913 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-17T08:32:13.913 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-17T08:32:13.941 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-17T08:32:13.942 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-17T08:32:13.971 DEBUG:teuthology.orchestra.run.smithi170:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-7862d192-74ce-11ef-bceb-c7b262605968@iscsi.iscsi.a.service 2024-09-17T08:32:14.015 INFO:tasks.cephadm:Adding prometheus.a on smithi170 2024-09-17T08:32:14.015 DEBUG:teuthology.orchestra.run.smithi170:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph orch apply prometheus '1;smithi170=a' 2024-09-17T08:32:14.743 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:32:14.863 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:14 smithi170 ceph-mon[33271]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.170", "placement": "1;smithi170=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:32:14.863 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:14 smithi170 ceph-mon[33271]: Saving service iscsi.datapool spec with placement smithi170=iscsi.a;count:1 2024-09-17T08:32:14.863 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:14 smithi170 ceph-mon[33271]: pgmap v194: 132 pgs: 1 creating+activating, 14 creating+peering, 38 unknown, 79 active+clean; 578 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 767 B/s wr, 1 op/s 2024-09-17T08:32:14.863 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:14 smithi170 ceph-mon[33271]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-17T08:32:14.863 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:14 smithi170 ceph-mon[33271]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-17T08:32:14.864 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:14 smithi170 ceph-mon[33271]: osdmap e67: 8 total, 8 up, 8 in 2024-09-17T08:32:14.864 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:14 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:14.864 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:14 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:14.864 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:14 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:14.864 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:14 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-17T08:32:14.864 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:14 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-17T08:32:14.865 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:14 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:32:15.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:14 smithi118 ceph-mon[29491]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.170", "placement": "1;smithi170=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:32:15.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:14 smithi118 ceph-mon[29491]: Saving service iscsi.datapool spec with placement smithi170=iscsi.a;count:1 2024-09-17T08:32:15.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:14 smithi118 ceph-mon[29491]: pgmap v194: 132 pgs: 1 creating+activating, 14 creating+peering, 38 unknown, 79 active+clean; 578 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 767 B/s wr, 1 op/s 2024-09-17T08:32:15.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:14 smithi118 ceph-mon[29491]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-17T08:32:15.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:14 smithi118 ceph-mon[29491]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-17T08:32:15.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:14 smithi118 ceph-mon[29491]: osdmap e67: 8 total, 8 up, 8 in 2024-09-17T08:32:15.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:14 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:15.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:14 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:15.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:14 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:15.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:14 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-17T08:32:15.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:14 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-17T08:32:15.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:14 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:32:15.029 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:14 smithi118 ceph-mon[41418]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.170", "placement": "1;smithi170=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:32:15.029 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:14 smithi118 ceph-mon[41418]: Saving service iscsi.datapool spec with placement smithi170=iscsi.a;count:1 2024-09-17T08:32:15.029 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:14 smithi118 ceph-mon[41418]: pgmap v194: 132 pgs: 1 creating+activating, 14 creating+peering, 38 unknown, 79 active+clean; 578 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 767 B/s wr, 1 op/s 2024-09-17T08:32:15.029 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:14 smithi118 ceph-mon[41418]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-17T08:32:15.029 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:14 smithi118 ceph-mon[41418]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-17T08:32:15.029 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:14 smithi118 ceph-mon[41418]: osdmap e67: 8 total, 8 up, 8 in 2024-09-17T08:32:15.029 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:14 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:15.029 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:14 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:15.029 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:14 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:15.029 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:14 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-17T08:32:15.029 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:14 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-17T08:32:15.030 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:14 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:32:15.623 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:15 smithi170 ceph-mon[33271]: Deploying daemon iscsi.iscsi.a on smithi170 2024-09-17T08:32:15.895 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:32:15 smithi170 systemd[1]: Starting Ceph iscsi.iscsi.a for 7862d192-74ce-11ef-bceb-c7b262605968... 2024-09-17T08:32:16.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:15 smithi118 ceph-mon[29491]: Deploying daemon iscsi.iscsi.a on smithi170 2024-09-17T08:32:16.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:15 smithi118 ceph-mon[41418]: Deploying daemon iscsi.iscsi.a on smithi170 2024-09-17T08:32:16.541 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:32:16 smithi170 podman[61819]: 2024-09-17 08:32:16.280221022 +0000 UTC m=+0.018886270 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T08:32:16.541 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:32:16 smithi170 podman[61819]: 2024-09-17 08:32:16.424145574 +0000 UTC m=+0.162810813 container create 381796f80abeac7102a9ba74dc06e379958c590ba8b1191e302c14cf47cfaaf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a-tcmu, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=reef-19e2a2b, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T08:32:16.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:16 smithi170 ceph-mon[33271]: pgmap v196: 132 pgs: 1 creating+activating, 9 creating+peering, 11 unknown, 111 active+clean; 578 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 1.4 KiB/s wr, 27 op/s 2024-09-17T08:32:16.896 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:32:16 smithi170 podman[61819]: 2024-09-17 08:32:16.643259141 +0000 UTC m=+0.381924388 container init 381796f80abeac7102a9ba74dc06e379958c590ba8b1191e302c14cf47cfaaf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a-tcmu, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, RELEASE=reef-19e2a2b, CEPH_POINT_RELEASE=) 2024-09-17T08:32:16.896 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:32:16 smithi170 podman[61819]: 2024-09-17 08:32:16.648024193 +0000 UTC m=+0.386689436 container start 381796f80abeac7102a9ba74dc06e379958c590ba8b1191e302c14cf47cfaaf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a-tcmu, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-17T08:32:16.896 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:32:16 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a-tcmu[61874]: log file path now is '/var/log/tcmu-runner.log' 2024-09-17T08:32:16.896 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:32:16 smithi170 bash[61819]: 381796f80abeac7102a9ba74dc06e379958c590ba8b1191e302c14cf47cfaaf5 2024-09-17T08:32:17.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:16 smithi118 ceph-mon[29491]: pgmap v196: 132 pgs: 1 creating+activating, 9 creating+peering, 11 unknown, 111 active+clean; 578 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 1.4 KiB/s wr, 27 op/s 2024-09-17T08:32:17.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:16 smithi118 ceph-mon[41418]: pgmap v196: 132 pgs: 1 creating+activating, 9 creating+peering, 11 unknown, 111 active+clean; 578 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 1.4 KiB/s wr, 27 op/s 2024-09-17T08:32:17.527 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:32:17 smithi170 podman[61983]: 2024-09-17 08:32:17.447532553 +0000 UTC m=+0.104590903 container create 67e86fcc3bc37e17cf7fdbe7100de8d1ae0f996b42bfc67e7eba6405d98d3c4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a, RELEASE=reef-19e2a2b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-17T08:32:17.527 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:32:17 smithi170 podman[61983]: 2024-09-17 08:32:17.360182816 +0000 UTC m=+0.017241181 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T08:32:17.528 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled prometheus update... 2024-09-17T08:32:18.145 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:32:17 smithi170 podman[61983]: 2024-09-17 08:32:17.859413375 +0000 UTC m=+0.516471735 container init 67e86fcc3bc37e17cf7fdbe7100de8d1ae0f996b42bfc67e7eba6405d98d3c4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-17T08:32:18.145 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:32:17 smithi170 podman[61983]: 2024-09-17 08:32:17.864769564 +0000 UTC m=+0.521827916 container start 67e86fcc3bc37e17cf7fdbe7100de8d1ae0f996b42bfc67e7eba6405d98d3c4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, RELEASE=reef-19e2a2b, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T08:32:18.146 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:32:17 smithi170 bash[61983]: 67e86fcc3bc37e17cf7fdbe7100de8d1ae0f996b42bfc67e7eba6405d98d3c4a 2024-09-17T08:32:18.146 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:32:17 smithi170 systemd[1]: Started Ceph iscsi.iscsi.a for 7862d192-74ce-11ef-bceb-c7b262605968. 2024-09-17T08:32:18.415 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:32:18 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug Started the configuration object watcher 2024-09-17T08:32:18.415 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:32:18 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug Checking for config object changes every 1s 2024-09-17T08:32:18.415 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:32:18 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug Processing osd blocklist entries for this node 2024-09-17T08:32:18.502 DEBUG:teuthology.orchestra.run.smithi170:prometheus.a> sudo journalctl -f -n 0 -u ceph-7862d192-74ce-11ef-bceb-c7b262605968@prometheus.a.service 2024-09-17T08:32:18.504 INFO:tasks.cephadm:Adding node-exporter.a on smithi118 2024-09-17T08:32:18.505 INFO:tasks.cephadm:Adding node-exporter.b on smithi170 2024-09-17T08:32:18.505 DEBUG:teuthology.orchestra.run.smithi170:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph orch apply node-exporter '2;smithi118=a;smithi170=b' 2024-09-17T08:32:18.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:18 smithi118 ceph-mon[29491]: pgmap v197: 132 pgs: 1 creating+activating, 5 creating+peering, 126 active+clean; 579 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 28 KiB/s rd, 2.5 KiB/s wr, 69 op/s 2024-09-17T08:32:18.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:18 smithi118 ceph-mon[29491]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi170=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:32:18.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:18 smithi118 ceph-mon[29491]: Saving service prometheus spec with placement smithi170=a;count:1 2024-09-17T08:32:18.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:18 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:18.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:18 smithi118 ceph-mon[29491]: osdmap e68: 8 total, 8 up, 8 in 2024-09-17T08:32:18.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:18 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:18.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:18 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:18.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:18 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:18.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:18 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:18.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:18 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:32:18.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:18 smithi118 ceph-mon[29491]: from='client.? 172.21.15.170:0/286451487' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-17T08:32:18.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:18 smithi118 ceph-mon[41418]: pgmap v197: 132 pgs: 1 creating+activating, 5 creating+peering, 126 active+clean; 579 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 28 KiB/s rd, 2.5 KiB/s wr, 69 op/s 2024-09-17T08:32:18.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:18 smithi118 ceph-mon[41418]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi170=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:32:18.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:18 smithi118 ceph-mon[41418]: Saving service prometheus spec with placement smithi170=a;count:1 2024-09-17T08:32:18.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:18 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:18.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:18 smithi118 ceph-mon[41418]: osdmap e68: 8 total, 8 up, 8 in 2024-09-17T08:32:18.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:18 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:18.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:18 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:18.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:18 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:18.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:18 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:18.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:18 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:32:18.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:18 smithi118 ceph-mon[41418]: from='client.? 172.21.15.170:0/286451487' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-17T08:32:18.872 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:18 smithi170 ceph-mon[33271]: pgmap v197: 132 pgs: 1 creating+activating, 5 creating+peering, 126 active+clean; 579 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 28 KiB/s rd, 2.5 KiB/s wr, 69 op/s 2024-09-17T08:32:18.872 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:18 smithi170 ceph-mon[33271]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi170=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:32:18.872 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:18 smithi170 ceph-mon[33271]: Saving service prometheus spec with placement smithi170=a;count:1 2024-09-17T08:32:18.873 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:18 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:18.873 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:18 smithi170 ceph-mon[33271]: osdmap e68: 8 total, 8 up, 8 in 2024-09-17T08:32:18.873 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:18 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:18.873 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:18 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:18.873 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:18 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:18.873 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:18 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:18.873 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:18 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:32:18.873 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:18 smithi170 ceph-mon[33271]: from='client.? 172.21.15.170:0/286451487' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-17T08:32:18.873 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:32:18 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug Reading the configuration object to update local LIO configuration 2024-09-17T08:32:18.873 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:32:18 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug Configuration does not have an entry for this host(smithi170.front.sepia.ceph.com) - nothing to define to LIO 2024-09-17T08:32:18.874 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:32:18 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: * Serving Flask app 'rbd-target-api' (lazy loading) 2024-09-17T08:32:18.874 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:32:18 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: * Environment: production 2024-09-17T08:32:18.874 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:32:18 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-17T08:32:18.874 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:32:18 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: Use a production WSGI server instead. 2024-09-17T08:32:18.874 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:32:18 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: * Debug mode: off 2024-09-17T08:32:18.874 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:32:18 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug * Running on all addresses. 2024-09-17T08:32:18.874 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:32:18 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-17T08:32:18.874 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:32:18 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: * Running on all addresses. 2024-09-17T08:32:18.874 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:32:18 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-17T08:32:18.874 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:32:18 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug * Running on http://[fe80::ec4:7aff:fe88:7255]:5000/ (Press CTRL+C to quit) 2024-09-17T08:32:18.874 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:32:18 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: * Running on http://[fe80::ec4:7aff:fe88:7255]:5000/ (Press CTRL+C to quit) 2024-09-17T08:32:19.328 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:32:19.710 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:19 smithi170 ceph-mon[33271]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-17T08:32:20.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:19 smithi118 ceph-mon[29491]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-17T08:32:20.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:19 smithi118 ceph-mon[41418]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-17T08:32:20.767 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:20 smithi170 ceph-mon[33271]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 67 KiB/s rd, 5.4 KiB/s wr, 165 op/s 2024-09-17T08:32:20.767 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:20 smithi170 ceph-mon[33271]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-17T08:32:21.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:20 smithi118 ceph-mon[29491]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 67 KiB/s rd, 5.4 KiB/s wr, 165 op/s 2024-09-17T08:32:21.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:20 smithi118 ceph-mon[29491]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-17T08:32:21.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:20 smithi118 ceph-mon[41418]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 67 KiB/s rd, 5.4 KiB/s wr, 165 op/s 2024-09-17T08:32:21.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:20 smithi118 ceph-mon[41418]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-17T08:32:21.206 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled node-exporter update... 2024-09-17T08:32:21.909 DEBUG:teuthology.orchestra.run.smithi118:node-exporter.a> sudo journalctl -f -n 0 -u ceph-7862d192-74ce-11ef-bceb-c7b262605968@node-exporter.a.service 2024-09-17T08:32:21.912 DEBUG:teuthology.orchestra.run.smithi170:node-exporter.b> sudo journalctl -f -n 0 -u ceph-7862d192-74ce-11ef-bceb-c7b262605968@node-exporter.b.service 2024-09-17T08:32:21.915 INFO:tasks.cephadm:Adding alertmanager.a on smithi118 2024-09-17T08:32:21.915 DEBUG:teuthology.orchestra.run.smithi170:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph orch apply alertmanager '1;smithi118=a' 2024-09-17T08:32:22.374 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:32:22.494 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:22 smithi170 ceph-mon[33271]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi118=a;smithi170=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:32:22.494 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:22 smithi170 ceph-mon[33271]: Saving service node-exporter spec with placement smithi118=a;smithi170=b;count:2 2024-09-17T08:32:22.494 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:22 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:22.495 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:22 smithi170 ceph-mon[33271]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 4.6 KiB/s wr, 141 op/s 2024-09-17T08:32:22.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:22 smithi118 ceph-mon[41418]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi118=a;smithi170=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:32:22.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:22 smithi118 ceph-mon[41418]: Saving service node-exporter spec with placement smithi118=a;smithi170=b;count:2 2024-09-17T08:32:22.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:22 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:22.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:22 smithi118 ceph-mon[41418]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 4.6 KiB/s wr, 141 op/s 2024-09-17T08:32:22.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:22 smithi118 ceph-mon[29491]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi118=a;smithi170=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:32:22.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:22 smithi118 ceph-mon[29491]: Saving service node-exporter spec with placement smithi118=a;smithi170=b;count:2 2024-09-17T08:32:22.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:22 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:22.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:22 smithi118 ceph-mon[29491]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 4.6 KiB/s wr, 141 op/s 2024-09-17T08:32:23.564 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:23 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:23.565 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:23 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:23.565 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:23 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:32:23.565 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:23 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:32:23.565 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:23 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:23.565 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:23 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:23.565 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:23 smithi170 ceph-mon[33271]: Deploying daemon prometheus.a on smithi170 2024-09-17T08:32:23.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:23 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:23.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:23 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:23.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:23 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:32:23.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:23 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:32:23.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:23 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:23.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:23 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:23.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:23 smithi118 ceph-mon[29491]: Deploying daemon prometheus.a on smithi170 2024-09-17T08:32:23.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:23 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:23.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:23 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:23.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:23 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:32:23.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:23 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:32:23.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:23 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:23.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:23 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:23.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:23 smithi118 ceph-mon[41418]: Deploying daemon prometheus.a on smithi170 2024-09-17T08:32:24.256 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled alertmanager update... 2024-09-17T08:32:24.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:24 smithi170 ceph-mon[33271]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.7 KiB/s wr, 114 op/s 2024-09-17T08:32:24.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:24 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:24.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:24 smithi118 ceph-mon[29491]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.7 KiB/s wr, 114 op/s 2024-09-17T08:32:24.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:24 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:24.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:24 smithi118 ceph-mon[41418]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.7 KiB/s wr, 114 op/s 2024-09-17T08:32:24.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:24 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:25.044 DEBUG:teuthology.orchestra.run.smithi118:alertmanager.a> sudo journalctl -f -n 0 -u ceph-7862d192-74ce-11ef-bceb-c7b262605968@alertmanager.a.service 2024-09-17T08:32:25.047 INFO:tasks.cephadm:Adding grafana.a on smithi170 2024-09-17T08:32:25.048 DEBUG:teuthology.orchestra.run.smithi170:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph orch apply grafana '1;smithi170=a' 2024-09-17T08:32:25.247 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:32:25.634 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:25 smithi170 ceph-mon[33271]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi118=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:32:25.634 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:25 smithi170 ceph-mon[33271]: Saving service alertmanager spec with placement smithi118=a;count:1 2024-09-17T08:32:25.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:25 smithi118 ceph-mon[29491]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi118=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:32:25.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:25 smithi118 ceph-mon[29491]: Saving service alertmanager spec with placement smithi118=a;count:1 2024-09-17T08:32:25.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:25 smithi118 ceph-mon[41418]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi118=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:32:25.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:25 smithi118 ceph-mon[41418]: Saving service alertmanager spec with placement smithi118=a;count:1 2024-09-17T08:32:26.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:26 smithi170 ceph-mon[33271]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 41 KiB/s rd, 3.2 KiB/s wr, 100 op/s 2024-09-17T08:32:26.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:26 smithi118 ceph-mon[29491]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 41 KiB/s rd, 3.2 KiB/s wr, 100 op/s 2024-09-17T08:32:26.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:26 smithi118 ceph-mon[41418]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 41 KiB/s rd, 3.2 KiB/s wr, 100 op/s 2024-09-17T08:32:27.901 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled grafana update... 2024-09-17T08:32:28.645 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:32:28 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:32:28.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:28 smithi170 ceph-mon[33271]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 2.4 KiB/s wr, 71 op/s 2024-09-17T08:32:28.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:28 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:28.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:28 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:28.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:28 smithi118 ceph-mon[29491]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 2.4 KiB/s wr, 71 op/s 2024-09-17T08:32:28.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:28 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:28.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:28 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:28.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:28 smithi118 ceph-mon[41418]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 2.4 KiB/s wr, 71 op/s 2024-09-17T08:32:28.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:28 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:28.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:28 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:29.655 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:29 smithi170 ceph-mon[33271]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi170=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:32:29.655 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:29 smithi170 ceph-mon[33271]: Saving service grafana spec with placement smithi170=a;count:1 2024-09-17T08:32:29.655 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:29 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:32:29.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:29 smithi118 ceph-mon[29491]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi170=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:32:29.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:29 smithi118 ceph-mon[29491]: Saving service grafana spec with placement smithi170=a;count:1 2024-09-17T08:32:29.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:29 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:32:29.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:29 smithi118 ceph-mon[41418]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi170=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:32:29.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:29 smithi118 ceph-mon[41418]: Saving service grafana spec with placement smithi170=a;count:1 2024-09-17T08:32:29.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:29 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:32:30.213 DEBUG:teuthology.orchestra.run.smithi170:grafana.a> sudo journalctl -f -n 0 -u ceph-7862d192-74ce-11ef-bceb-c7b262605968@grafana.a.service 2024-09-17T08:32:30.216 INFO:tasks.cephadm:Setting up client nodes... 2024-09-17T08:32:30.216 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-17T08:32:30.422 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:32:30.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:30 smithi118 ceph-mon[29491]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 885 B/s wr, 23 op/s 2024-09-17T08:32:30.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:30 smithi118 ceph-mon[41418]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 885 B/s wr, 23 op/s 2024-09-17T08:32:30.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:30 smithi170 ceph-mon[33271]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 885 B/s wr, 23 op/s 2024-09-17T08:32:32.429 INFO:teuthology.orchestra.run.smithi118.stdout:[client.0] 2024-09-17T08:32:32.429 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQCgPulmYt1RGRAAqcfhsvdOwR9u+TCRgzAndg== 2024-09-17T08:32:32.456 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:32 smithi118 ceph-mon[29491]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:32:32.456 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:32 smithi118 ceph-mon[41418]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:32:32.512 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:32 smithi170 ceph-mon[33271]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:32:33.468 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-17T08:32:33.468 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-17T08:32:33.468 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-17T08:32:33.506 DEBUG:teuthology.orchestra.run.smithi170:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-17T08:32:33.535 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:33 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/2222670754' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T08:32:33.535 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:33 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/2222670754' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T08:32:33.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:33 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/2222670754' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T08:32:33.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:33 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/2222670754' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T08:32:33.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:33 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/2222670754' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T08:32:33.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:33 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/2222670754' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T08:32:34.157 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:32:34.537 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:34 smithi170 ceph-mon[33271]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:32:34.538 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:32:34 smithi170 systemd[1]: Starting Ceph prometheus.a for 7862d192-74ce-11ef-bceb-c7b262605968... 2024-09-17T08:32:34.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:34 smithi118 ceph-mon[29491]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:32:34.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:34 smithi118 ceph-mon[41418]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:32:36.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:36 smithi118 ceph-mon[29491]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:32:36.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:36 smithi118 ceph-mon[41418]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:32:36.824 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:36 smithi170 ceph-mon[33271]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:32:37.619 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:37 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-09-17T08:32:37.620 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:37 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2024-09-17T08:32:37.620 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:37 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-09-17T08:32:37.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:37 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-09-17T08:32:37.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:37 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2024-09-17T08:32:37.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:37 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-09-17T08:32:37.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:37 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-09-17T08:32:37.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:37 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2024-09-17T08:32:37.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:37 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-09-17T08:32:37.891 INFO:teuthology.orchestra.run.smithi170.stdout:[client.1] 2024-09-17T08:32:37.892 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQClPulmPXbTNBAAYZsNNAl2zHdRk3802YAh5Q== 2024-09-17T08:32:37.892 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:32:37 smithi170 podman[63889]: 2024-09-17 08:32:37.633863901 +0000 UTC m=+0.247438224 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-17T08:32:37.892 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:32:37 smithi170 podman[63889]: 2024-09-17 08:32:37.737457291 +0000 UTC m=+0.351031604 container create 2b28ec08861ac5156a4366c4b4921a2fbf4bc4819d11e4c130e286fc6499a18f (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T08:32:38.146 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:32:37 smithi170 podman[63889]: 2024-09-17 08:32:37.998549225 +0000 UTC m=+0.612123544 container init 2b28ec08861ac5156a4366c4b4921a2fbf4bc4819d11e4c130e286fc6499a18f (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T08:32:38.146 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:32:38 smithi170 podman[63889]: 2024-09-17 08:32:38.003778958 +0000 UTC m=+0.617353271 container start 2b28ec08861ac5156a4366c4b4921a2fbf4bc4819d11e4c130e286fc6499a18f (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T08:32:38.146 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:32:38 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[63929]: ts=2024-09-17T08:32:38.028Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-17T08:32:38.146 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:32:38 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[63929]: ts=2024-09-17T08:32:38.028Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-17T08:32:38.146 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:32:38 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[63929]: ts=2024-09-17T08:32:38.028Z caller=main.go:570 level=info host_details="(Linux 5.14.0-508.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 12 15:49:37 UTC 2024 x86_64 smithi170 (none))" 2024-09-17T08:32:38.146 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:32:38 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[63929]: ts=2024-09-17T08:32:38.028Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-17T08:32:38.146 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:32:38 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[63929]: ts=2024-09-17T08:32:38.028Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-17T08:32:38.146 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:32:38 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[63929]: ts=2024-09-17T08:32:38.029Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-17T08:32:38.146 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:32:38 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[63929]: ts=2024-09-17T08:32:38.029Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-17T08:32:38.147 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:32:38 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[63929]: ts=2024-09-17T08:32:38.030Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-17T08:32:38.147 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:32:38 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[63929]: ts=2024-09-17T08:32:38.030Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-17T08:32:38.147 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:32:38 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[63929]: ts=2024-09-17T08:32:38.033Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-17T08:32:38.147 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:32:38 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[63929]: ts=2024-09-17T08:32:38.033Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=3.01µs 2024-09-17T08:32:38.147 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:32:38 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[63929]: ts=2024-09-17T08:32:38.033Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-17T08:32:38.147 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:32:38 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[63929]: ts=2024-09-17T08:32:38.033Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-09-17T08:32:38.147 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:32:38 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[63929]: ts=2024-09-17T08:32:38.033Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=106.926µs wal_replay_duration=393.965µs wbl_replay_duration=82ns total_replay_duration=547.358µs 2024-09-17T08:32:38.147 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:32:38 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[63929]: ts=2024-09-17T08:32:38.034Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-17T08:32:38.147 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:32:38 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[63929]: ts=2024-09-17T08:32:38.034Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-17T08:32:38.147 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:32:38 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[63929]: ts=2024-09-17T08:32:38.034Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-17T08:32:38.147 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:32:38 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[63929]: ts=2024-09-17T08:32:38.054Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=20.388629ms db_storage=701ns remote_storage=1.511µs web_handler=309ns query_engine=530ns scrape=407.049µs scrape_sd=110.042µs notify=1.033µs notify_sd=2.071µs rules=19.568188ms tracing=7.689µs 2024-09-17T08:32:38.147 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:32:38 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[63929]: ts=2024-09-17T08:32:38.054Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-17T08:32:38.148 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:32:38 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[63929]: ts=2024-09-17T08:32:38.054Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-17T08:32:38.148 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:32:38 smithi170 bash[63889]: 2b28ec08861ac5156a4366c4b4921a2fbf4bc4819d11e4c130e286fc6499a18f 2024-09-17T08:32:38.148 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:32:38 smithi170 systemd[1]: Started Ceph prometheus.a for 7862d192-74ce-11ef-bceb-c7b262605968. 2024-09-17T08:32:38.443 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:32:38 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:32:38.728 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:38 smithi170 ceph-mon[33271]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:32:38.729 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:38 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]': finished 2024-09-17T08:32:38.729 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:38 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]': finished 2024-09-17T08:32:38.729 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:38 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]': finished 2024-09-17T08:32:38.729 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:38 smithi170 ceph-mon[33271]: osdmap e69: 8 total, 8 up, 8 in 2024-09-17T08:32:38.729 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:38 smithi170 ceph-mon[33271]: from='client.? 172.21.15.170:0/1931430103' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T08:32:38.729 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:38 smithi170 ceph-mon[33271]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T08:32:38.730 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:38 smithi170 ceph-mon[33271]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T08:32:38.730 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:38 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:38.730 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:38 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:38.730 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:38 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:38.730 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:38 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-17T08:32:38.750 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-17T08:32:38.750 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-17T08:32:38.750 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-17T08:32:38.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:38 smithi118 ceph-mon[29491]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:32:38.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:38 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]': finished 2024-09-17T08:32:38.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:38 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]': finished 2024-09-17T08:32:38.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:38 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]': finished 2024-09-17T08:32:38.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:38 smithi118 ceph-mon[29491]: osdmap e69: 8 total, 8 up, 8 in 2024-09-17T08:32:38.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:38 smithi118 ceph-mon[29491]: from='client.? 172.21.15.170:0/1931430103' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T08:32:38.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:38 smithi118 ceph-mon[29491]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T08:32:38.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:38 smithi118 ceph-mon[29491]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T08:32:38.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:38 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:38.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:38 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:38.779 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:38 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:38.780 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:38 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-17T08:32:38.780 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:38 smithi118 ceph-mon[41418]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:32:38.780 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:38 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]': finished 2024-09-17T08:32:38.780 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:38 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]': finished 2024-09-17T08:32:38.781 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:38 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]': finished 2024-09-17T08:32:38.781 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:38 smithi118 ceph-mon[41418]: osdmap e69: 8 total, 8 up, 8 in 2024-09-17T08:32:38.781 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:38 smithi118 ceph-mon[41418]: from='client.? 172.21.15.170:0/1931430103' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T08:32:38.781 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:38 smithi118 ceph-mon[41418]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T08:32:38.781 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:38 smithi118 ceph-mon[41418]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T08:32:38.781 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:38 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:38.782 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:38 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:38.782 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:38 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' 2024-09-17T08:32:38.782 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:38 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-17T08:32:38.829 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-17T08:32:38.829 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-17T08:32:38.829 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-17T08:32:39.034 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:32:39.411 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:39 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ignoring --setuser ceph since I am not root 2024-09-17T08:32:39.412 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:39 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ignoring --setgroup ceph since I am not root 2024-09-17T08:32:39.412 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:39 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:32:39.318+0000 7fce85b84040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T08:32:39.412 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:39 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:32:39.375+0000 7fce85b84040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T08:32:39.523 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:39 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:32:39.524 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:39 smithi170 ceph-mon[33271]: osdmap e70: 8 total, 8 up, 8 in 2024-09-17T08:32:39.524 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:39 smithi170 ceph-mon[33271]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-17T08:32:39.524 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:39 smithi170 ceph-mon[33271]: mgrmap e17: y(active, since 5m), standbys: x 2024-09-17T08:32:39.524 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:32:39 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: ignoring --setuser ceph since I am not root 2024-09-17T08:32:39.524 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:32:39 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: ignoring --setgroup ceph since I am not root 2024-09-17T08:32:39.525 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:32:39 smithi170 ceph-mgr[35374]: -- 172.21.15.170:0/268295507 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56047b97cd00 con 0x56047b951000 2024-09-17T08:32:39.525 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:32:39 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:32:39.320+0000 7fa7363fa040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T08:32:39.525 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:32:39 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:32:39.376+0000 7fa7363fa040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T08:32:39.525 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:32:39 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:32:39.523+0000 7fa7363fa040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T08:32:39.735 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:39 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:32:39.735 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:39 smithi118 ceph-mon[29491]: osdmap e70: 8 total, 8 up, 8 in 2024-09-17T08:32:39.735 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:39 smithi118 ceph-mon[29491]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-17T08:32:39.735 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:39 smithi118 ceph-mon[29491]: mgrmap e17: y(active, since 5m), standbys: x 2024-09-17T08:32:39.736 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:39 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:32:39.528+0000 7fce85b84040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T08:32:39.736 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:39 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:32:39.592+0000 7fce85b84040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T08:32:39.736 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:39 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:32:39.736 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:39 smithi118 ceph-mon[41418]: osdmap e70: 8 total, 8 up, 8 in 2024-09-17T08:32:39.736 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:39 smithi118 ceph-mon[41418]: from='mgr.14150 172.21.15.118:0/1502477407' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-17T08:32:39.736 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:39 smithi118 ceph-mon[41418]: mgrmap e17: y(active, since 5m), standbys: x 2024-09-17T08:32:39.870 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:32:39 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:32:39.588+0000 7fa7363fa040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T08:32:39.870 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:32:39 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:32:39.731+0000 7fa7363fa040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:32:39.870 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:32:39 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:32:39.869+0000 7fa7363fa040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T08:32:40.027 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:39 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:32:39.745+0000 7fce85b84040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:32:40.027 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:39 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:32:39.884+0000 7fce85b84040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T08:32:40.439 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:32:40.195+0000 7fce85b84040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T08:32:40.440 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:32:40.256+0000 7fce85b84040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T08:32:40.645 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:32:40 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:32:40.176+0000 7fa7363fa040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T08:32:40.645 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:32:40 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:32:40.236+0000 7fa7363fa040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T08:32:40.645 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:32:40 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:32:40.416+0000 7fa7363fa040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T08:32:40.707 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:32:40.438+0000 7fce85b84040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T08:32:41.278 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:32:41.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:41 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/782159046' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-17T08:32:41.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:41 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/782159046' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-17T08:32:41.528 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:41 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:32:41.340+0000 7fce85b84040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T08:32:41.528 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:41 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:32:41.407+0000 7fce85b84040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T08:32:41.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:41 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/782159046' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-17T08:32:41.645 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:32:41 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:32:41.278+0000 7fa7363fa040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T08:32:41.645 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:32:41 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:32:41.344+0000 7fa7363fa040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T08:32:42.028 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":2556545142},{"type":"v1","addr":"172.21.15.118:6801","nonce":2556545142}]},"active_addr":"172.21.15.118:6801/2556545142","active_change":"2024-09-17T08:27:37.329542+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24115,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.118:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.118:0","nonce":3471266426}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.118:0","nonce":1555118856}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.118:0","nonce":1111095991}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.118:0","nonce":345247749}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.118:0","nonce":4056359646}]}]} 2024-09-17T08:32:42.035 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-17T08:32:42.036 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-17T08:32:42.036 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-17T08:32:42.239 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:32:42.277 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:42 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:32:42.126+0000 7fce85b84040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T08:32:42.277 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:42 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:32:42.253+0000 7fce85b84040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T08:32:42.292 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:32:42 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:32:42.039+0000 7fa7363fa040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T08:32:42.292 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:32:42 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:32:42.153+0000 7fa7363fa040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T08:32:42.567 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:32:42 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:32:42.291+0000 7fa7363fa040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:32:42.567 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:32:42 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:32:42.353+0000 7fa7363fa040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T08:32:42.567 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:32:42 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:32:42.497+0000 7fa7363fa040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T08:32:42.567 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:32:42 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:32:42.566+0000 7fa7363fa040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T08:32:42.613 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:42 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:32:42.398+0000 7fce85b84040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:32:42.614 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:42 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:32:42.458+0000 7fce85b84040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T08:32:42.845 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:32:42 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:32:42.626+0000 7fa7363fa040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T08:32:42.845 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:32:42 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:32:42.780+0000 7fa7363fa040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T08:32:42.901 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:42 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:32:42.613+0000 7fce85b84040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T08:32:42.901 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:42 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:32:42.679+0000 7fce85b84040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T08:32:42.902 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:42 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:32:42.742+0000 7fce85b84040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T08:32:42.902 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:42 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:32:42.901+0000 7fce85b84040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T08:32:43.108 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:32:42 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:32:42.844+0000 7fa7363fa040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T08:32:43.109 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:32:42 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:32:42.909+0000 7fa7363fa040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T08:32:43.109 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:32:42 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:32:42.991+0000 7fa7363fa040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T08:32:43.109 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:32:43 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:32:43.050+0000 7fa7363fa040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T08:32:43.109 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:32:43 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:32:43.108+0000 7fa7363fa040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T08:32:43.174 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:42 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:32:42.968+0000 7fce85b84040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T08:32:43.174 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:43 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:32:43.032+0000 7fce85b84040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T08:32:43.174 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:43 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:32:43.113+0000 7fce85b84040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T08:32:43.481 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:43 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:32:43.173+0000 7fce85b84040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T08:32:43.481 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:43 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:32:43.232+0000 7fce85b84040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T08:32:44.106 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:32:44.036+0000 7fce85b84040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T08:32:44.106 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:43 smithi118 ceph-mon[29491]: Standby manager daemon x restarted 2024-09-17T08:32:44.106 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:43 smithi118 ceph-mon[29491]: Standby manager daemon x started 2024-09-17T08:32:44.106 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:43 smithi118 ceph-mon[29491]: from='mgr.? 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-17T08:32:44.106 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:43 smithi118 ceph-mon[29491]: from='mgr.? 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T08:32:44.106 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:43 smithi118 ceph-mon[29491]: from='mgr.? 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-17T08:32:44.107 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:43 smithi118 ceph-mon[29491]: from='mgr.? 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T08:32:44.107 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:43 smithi118 ceph-mon[41418]: Standby manager daemon x restarted 2024-09-17T08:32:44.107 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:43 smithi118 ceph-mon[41418]: Standby manager daemon x started 2024-09-17T08:32:44.107 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:43 smithi118 ceph-mon[41418]: from='mgr.? 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-17T08:32:44.107 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:43 smithi118 ceph-mon[41418]: from='mgr.? 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T08:32:44.107 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:43 smithi118 ceph-mon[41418]: from='mgr.? 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-17T08:32:44.107 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:43 smithi118 ceph-mon[41418]: from='mgr.? 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T08:32:44.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:43 smithi170 ceph-mon[33271]: Standby manager daemon x restarted 2024-09-17T08:32:44.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:43 smithi170 ceph-mon[33271]: Standby manager daemon x started 2024-09-17T08:32:44.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:43 smithi170 ceph-mon[33271]: from='mgr.? 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-17T08:32:44.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:43 smithi170 ceph-mon[33271]: from='mgr.? 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T08:32:44.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:43 smithi170 ceph-mon[33271]: from='mgr.? 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-17T08:32:44.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:43 smithi170 ceph-mon[33271]: from='mgr.? 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T08:32:44.146 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:32:43 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:32:43.874+0000 7fa7363fa040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T08:32:44.146 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:32:43 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: [17/Sep/2024:08:32:43] ENGINE Bus STARTING 2024-09-17T08:32:44.146 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:32:43 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: CherryPy Checker: 2024-09-17T08:32:44.146 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:32:43 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: The Application mounted at '' has an empty config. 2024-09-17T08:32:44.146 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:32:43 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:32:44.146 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:32:43 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: [17/Sep/2024:08:32:43] ENGINE Serving on http://:::9283 2024-09-17T08:32:44.146 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:32:43 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: [17/Sep/2024:08:32:43] ENGINE Bus STARTED 2024-09-17T08:32:44.415 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:32:44.415 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: [17/Sep/2024:08:32:44] ENGINE Bus STARTING 2024-09-17T08:32:44.415 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: CherryPy Checker: 2024-09-17T08:32:44.415 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: The Application mounted at '' has an empty config. 2024-09-17T08:32:44.415 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:32:44.415 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: [17/Sep/2024:08:32:44] ENGINE Serving on http://:::9283 2024-09-17T08:32:44.415 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: [17/Sep/2024:08:32:44] ENGINE Bus STARTED 2024-09-17T08:32:44.416 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":71,"fsid":"7862d192-74ce-11ef-bceb-c7b262605968","created":"2024-09-17T08:26:42.865266+0000","modified":"2024-09-17T08:32:44.038810+0000","last_up_change":"2024-09-17T08:31:52.813568+0000","last_in_change":"2024-09-17T08:31:36.166866+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T08:29:53.628517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-17T08:32:05.136156+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":64,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-17T08:32:05.370518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-17T08:32:06.790686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-17T08:32:08.764713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-17T08:32:10.838104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ea41bec-a258-4d78-be6b-0299675f1106","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":3097740994},{"type":"v1","addr":"172.21.15.118:6803","nonce":3097740994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":3097740994},{"type":"v1","addr":"172.21.15.118:6805","nonce":3097740994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":3097740994},{"type":"v1","addr":"172.21.15.118:6809","nonce":3097740994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":3097740994},{"type":"v1","addr":"172.21.15.118:6807","nonce":3097740994}]},"public_addr":"172.21.15.118:6803/3097740994","cluster_addr":"172.21.15.118:6805/3097740994","heartbeat_back_addr":"172.21.15.118:6809/3097740994","heartbeat_front_addr":"172.21.15.118:6807/3097740994","state":["exists","up"]},{"osd":1,"uuid":"56073858-b245-4470-9941-1cdf67a69ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":3447428990},{"type":"v1","addr":"172.21.15.118:6811","nonce":3447428990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":3447428990},{"type":"v1","addr":"172.21.15.118:6813","nonce":3447428990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3447428990},{"type":"v1","addr":"172.21.15.118:6817","nonce":3447428990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":3447428990},{"type":"v1","addr":"172.21.15.118:6815","nonce":3447428990}]},"public_addr":"172.21.15.118:6811/3447428990","cluster_addr":"172.21.15.118:6813/3447428990","heartbeat_back_addr":"172.21.15.118:6817/3447428990","heartbeat_front_addr":"172.21.15.118:6815/3447428990","state":["exists","up"]},{"osd":2,"uuid":"edce62b8-f6c0-4c4a-b789-45304fa5eeb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1717873621},{"type":"v1","addr":"172.21.15.118:6819","nonce":1717873621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1717873621},{"type":"v1","addr":"172.21.15.118:6821","nonce":1717873621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1717873621},{"type":"v1","addr":"172.21.15.118:6825","nonce":1717873621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1717873621},{"type":"v1","addr":"172.21.15.118:6823","nonce":1717873621}]},"public_addr":"172.21.15.118:6819/1717873621","cluster_addr":"172.21.15.118:6821/1717873621","heartbeat_back_addr":"172.21.15.118:6825/1717873621","heartbeat_front_addr":"172.21.15.118:6823/1717873621","state":["exists","up"]},{"osd":3,"uuid":"1c7254e0-d2fd-4b11-a110-04f9d7aa04a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":1195800036},{"type":"v1","addr":"172.21.15.118:6827","nonce":1195800036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":1195800036},{"type":"v1","addr":"172.21.15.118:6829","nonce":1195800036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1195800036},{"type":"v1","addr":"172.21.15.118:6833","nonce":1195800036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":1195800036},{"type":"v1","addr":"172.21.15.118:6831","nonce":1195800036}]},"public_addr":"172.21.15.118:6827/1195800036","cluster_addr":"172.21.15.118:6829/1195800036","heartbeat_back_addr":"172.21.15.118:6833/1195800036","heartbeat_front_addr":"172.21.15.118:6831/1195800036","state":["exists","up"]},{"osd":4,"uuid":"46dcae68-591d-41d4-9829-ce25368caffa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1673072938},{"type":"v1","addr":"172.21.15.170:6801","nonce":1673072938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1673072938},{"type":"v1","addr":"172.21.15.170:6803","nonce":1673072938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1673072938},{"type":"v1","addr":"172.21.15.170:6807","nonce":1673072938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1673072938},{"type":"v1","addr":"172.21.15.170:6805","nonce":1673072938}]},"public_addr":"172.21.15.170:6801/1673072938","cluster_addr":"172.21.15.170:6803/1673072938","heartbeat_back_addr":"172.21.15.170:6807/1673072938","heartbeat_front_addr":"172.21.15.170:6805/1673072938","state":["exists","up"]},{"osd":5,"uuid":"d58f1343-ad2a-4153-89f7-e7d691aa5282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3749911609},{"type":"v1","addr":"172.21.15.170:6809","nonce":3749911609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3749911609},{"type":"v1","addr":"172.21.15.170:6811","nonce":3749911609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3749911609},{"type":"v1","addr":"172.21.15.170:6815","nonce":3749911609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3749911609},{"type":"v1","addr":"172.21.15.170:6813","nonce":3749911609}]},"public_addr":"172.21.15.170:6809/3749911609","cluster_addr":"172.21.15.170:6811/3749911609","heartbeat_back_addr":"172.21.15.170:6815/3749911609","heartbeat_front_addr":"172.21.15.170:6813/3749911609","state":["exists","up"]},{"osd":6,"uuid":"7b63464b-4cfc-4225-91a4-ca4fe84adeb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":965836527},{"type":"v1","addr":"172.21.15.170:6817","nonce":965836527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":965836527},{"type":"v1","addr":"172.21.15.170:6819","nonce":965836527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":965836527},{"type":"v1","addr":"172.21.15.170:6823","nonce":965836527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":965836527},{"type":"v1","addr":"172.21.15.170:6821","nonce":965836527}]},"public_addr":"172.21.15.170:6817/965836527","cluster_addr":"172.21.15.170:6819/965836527","heartbeat_back_addr":"172.21.15.170:6823/965836527","heartbeat_front_addr":"172.21.15.170:6821/965836527","state":["exists","up"]},{"osd":7,"uuid":"3431c0a5-527d-4199-b0e4-c17fc99f8981","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3980860619},{"type":"v1","addr":"172.21.15.170:6825","nonce":3980860619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3980860619},{"type":"v1","addr":"172.21.15.170:6827","nonce":3980860619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3980860619},{"type":"v1","addr":"172.21.15.170:6831","nonce":3980860619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3980860619},{"type":"v1","addr":"172.21.15.170:6829","nonce":3980860619}]},"public_addr":"172.21.15.170:6825/3980860619","cluster_addr":"172.21.15.170:6827/3980860619","heartbeat_back_addr":"172.21.15.170:6831/3980860619","heartbeat_front_addr":"172.21.15.170:6829/3980860619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:29:00.821543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:29:25.706259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:29:51.063464+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:30:17.959114+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:30:40.030347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:31:03.713849+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:31:27.429096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:31:51.600524+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.118:0/4056359646":"2024-09-18T08:32:44.038764+0000","172.21.15.118:0/3471266426":"2024-09-18T08:32:44.038764+0000","172.21.15.118:0/677224718":"2024-09-18T08:27:13.370329+0000","172.21.15.118:0/3611715606":"2024-09-18T08:27:13.370329+0000","172.21.15.118:0/1555118856":"2024-09-18T08:32:44.038764+0000","172.21.15.118:6800/862659270":"2024-09-18T08:27:13.370329+0000","172.21.15.118:6801/2556545142":"2024-09-18T08:32:44.038764+0000","172.21.15.118:6801/862659270":"2024-09-18T08:27:13.370329+0000","172.21.15.118:6800/2223255332":"2024-09-18T08:27:37.329435+0000","172.21.15.118:6801/2223255332":"2024-09-18T08:27:37.329435+0000","172.21.15.118:0/1111468500":"2024-09-18T08:27:37.329435+0000","172.21.15.118:6800/2556545142":"2024-09-18T08:32:44.038764+0000","172.21.15.118:0/2512398536":"2024-09-18T08:27:37.329435+0000","172.21.15.118:0/858276273":"2024-09-18T08:27:13.370329+0000","172.21.15.118:0/1798368975":"2024-09-18T08:27:37.329435+0000","172.21.15.118:0/1111095991":"2024-09-18T08:32:44.038764+0000","172.21.15.118:0/345247749":"2024-09-18T08:32:44.038764+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T08:32:45.158 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[29491]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-17T08:32:45.158 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[29491]: Active manager daemon y restarted 2024-09-17T08:32:45.158 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[29491]: Activating manager daemon y 2024-09-17T08:32:45.158 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[29491]: osdmap e71: 8 total, 8 up, 8 in 2024-09-17T08:32:45.158 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[29491]: mgrmap e19: y(active, starting, since 0.00718951s), standbys: x 2024-09-17T08:32:45.158 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:32:45.158 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:32:45.158 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:32:45.159 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-17T08:32:45.159 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-17T08:32:45.159 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:32:45.159 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:32:45.159 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:32:45.159 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:32:45.159 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:32:45.159 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:32:45.159 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:32:45.159 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:32:45.159 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-17T08:32:45.159 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-17T08:32:45.159 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-17T08:32:45.159 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[29491]: Manager daemon y is now available 2024-09-17T08:32:45.160 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:32:45.160 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:45.160 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:32:45.160 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-17T08:32:45.160 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-17T08:32:45.160 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-17T08:32:45.160 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-17T08:32:45.160 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/505245150' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T08:32:45.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[41418]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-17T08:32:45.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[41418]: Active manager daemon y restarted 2024-09-17T08:32:45.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[41418]: Activating manager daemon y 2024-09-17T08:32:45.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[41418]: osdmap e71: 8 total, 8 up, 8 in 2024-09-17T08:32:45.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[41418]: mgrmap e19: y(active, starting, since 0.00718951s), standbys: x 2024-09-17T08:32:45.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:32:45.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:32:45.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:32:45.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-17T08:32:45.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-17T08:32:45.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:32:45.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:32:45.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:32:45.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:32:45.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:32:45.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:32:45.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:32:45.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:32:45.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-17T08:32:45.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-17T08:32:45.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-17T08:32:45.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[41418]: Manager daemon y is now available 2024-09-17T08:32:45.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:32:45.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:45.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:32:45.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-17T08:32:45.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-17T08:32:45.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-17T08:32:45.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-17T08:32:45.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:44 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/505245150' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T08:32:45.178 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-17T08:32:45.178 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-17T08:32:45.282 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:44 smithi170 ceph-mon[33271]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-17T08:32:45.282 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:44 smithi170 ceph-mon[33271]: Active manager daemon y restarted 2024-09-17T08:32:45.282 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:44 smithi170 ceph-mon[33271]: Activating manager daemon y 2024-09-17T08:32:45.282 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:44 smithi170 ceph-mon[33271]: osdmap e71: 8 total, 8 up, 8 in 2024-09-17T08:32:45.282 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:44 smithi170 ceph-mon[33271]: mgrmap e19: y(active, starting, since 0.00718951s), standbys: x 2024-09-17T08:32:45.282 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:44 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:32:45.282 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:44 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:32:45.282 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:44 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:32:45.282 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:44 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-17T08:32:45.282 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:44 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-17T08:32:45.283 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:44 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:32:45.283 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:44 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:32:45.283 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:44 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:32:45.283 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:44 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:32:45.283 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:44 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:32:45.283 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:44 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:32:45.283 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:44 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:32:45.283 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:44 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:32:45.283 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:44 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-17T08:32:45.283 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:44 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-17T08:32:45.283 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:44 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-17T08:32:45.283 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:44 smithi170 ceph-mon[33271]: Manager daemon y is now available 2024-09-17T08:32:45.283 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:44 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:32:45.283 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:44 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:45.283 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:44 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:32:45.284 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:44 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-17T08:32:45.284 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:44 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-17T08:32:45.284 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:44 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-17T08:32:45.284 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:44 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-17T08:32:45.284 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:44 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/505245150' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T08:32:46.131 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:32:46.389 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:46 smithi118 ceph-mon[29491]: mgrmap e20: y(active, since 1.01031s), standbys: x 2024-09-17T08:32:46.389 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:46 smithi118 ceph-mon[29491]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:32:46.390 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:46 smithi118 ceph-mon[41418]: mgrmap e20: y(active, since 1.01031s), standbys: x 2024-09-17T08:32:46.390 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:46 smithi118 ceph-mon[41418]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:32:46.394 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:46 smithi170 ceph-mon[33271]: mgrmap e20: y(active, since 1.01031s), standbys: x 2024-09-17T08:32:46.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:46 smithi170 ceph-mon[33271]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:32:47.021 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:46 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:32:46.642+0000 7fcdda02b640 -1 log_channel(cephadm) log [ERR] : [17/Sep/2024:08:32:46] ENGINE Error in HTTPServer.serve 2024-09-17T08:32:47.021 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:46 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: Traceback (most recent call last): 2024-09-17T08:32:47.021 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:46 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T08:32:47.021 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:46 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: self._connections.run(self.expiration_interval) 2024-09-17T08:32:47.021 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:46 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T08:32:47.021 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:46 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: self._run(expiration_interval) 2024-09-17T08:32:47.022 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:46 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T08:32:47.022 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:46 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T08:32:47.022 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:46 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T08:32:47.022 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:46 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T08:32:47.022 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:46 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T08:32:47.022 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:46 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: s = self.context.wrap_socket( 2024-09-17T08:32:47.022 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:46 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T08:32:47.022 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:46 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: return self.sslsocket_class._create( 2024-09-17T08:32:47.022 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:46 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T08:32:47.022 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:46 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: self.do_handshake() 2024-09-17T08:32:47.022 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:46 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T08:32:47.022 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:46 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: self._sslobj.do_handshake() 2024-09-17T08:32:47.022 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:46 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T08:32:47.022 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:46 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:32:47.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:47 smithi118 ceph-mon[29491]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:32:47.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:47 smithi118 ceph-mon[29491]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-17T08:32:47.277 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:47 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:32:47] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:32:47.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:47 smithi118 ceph-mon[41418]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:32:47.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:47 smithi118 ceph-mon[41418]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-17T08:32:47.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:47 smithi170 ceph-mon[33271]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:32:47.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:47 smithi170 ceph-mon[33271]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-17T08:32:48.172 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:32:48.173 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":71,"fsid":"7862d192-74ce-11ef-bceb-c7b262605968","created":"2024-09-17T08:26:42.865266+0000","modified":"2024-09-17T08:32:44.038810+0000","last_up_change":"2024-09-17T08:31:52.813568+0000","last_in_change":"2024-09-17T08:31:36.166866+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T08:29:53.628517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-17T08:32:05.136156+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":64,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-17T08:32:05.370518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-17T08:32:06.790686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-17T08:32:08.764713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-17T08:32:10.838104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ea41bec-a258-4d78-be6b-0299675f1106","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":3097740994},{"type":"v1","addr":"172.21.15.118:6803","nonce":3097740994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":3097740994},{"type":"v1","addr":"172.21.15.118:6805","nonce":3097740994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":3097740994},{"type":"v1","addr":"172.21.15.118:6809","nonce":3097740994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":3097740994},{"type":"v1","addr":"172.21.15.118:6807","nonce":3097740994}]},"public_addr":"172.21.15.118:6803/3097740994","cluster_addr":"172.21.15.118:6805/3097740994","heartbeat_back_addr":"172.21.15.118:6809/3097740994","heartbeat_front_addr":"172.21.15.118:6807/3097740994","state":["exists","up"]},{"osd":1,"uuid":"56073858-b245-4470-9941-1cdf67a69ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":3447428990},{"type":"v1","addr":"172.21.15.118:6811","nonce":3447428990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":3447428990},{"type":"v1","addr":"172.21.15.118:6813","nonce":3447428990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3447428990},{"type":"v1","addr":"172.21.15.118:6817","nonce":3447428990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":3447428990},{"type":"v1","addr":"172.21.15.118:6815","nonce":3447428990}]},"public_addr":"172.21.15.118:6811/3447428990","cluster_addr":"172.21.15.118:6813/3447428990","heartbeat_back_addr":"172.21.15.118:6817/3447428990","heartbeat_front_addr":"172.21.15.118:6815/3447428990","state":["exists","up"]},{"osd":2,"uuid":"edce62b8-f6c0-4c4a-b789-45304fa5eeb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1717873621},{"type":"v1","addr":"172.21.15.118:6819","nonce":1717873621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1717873621},{"type":"v1","addr":"172.21.15.118:6821","nonce":1717873621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1717873621},{"type":"v1","addr":"172.21.15.118:6825","nonce":1717873621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1717873621},{"type":"v1","addr":"172.21.15.118:6823","nonce":1717873621}]},"public_addr":"172.21.15.118:6819/1717873621","cluster_addr":"172.21.15.118:6821/1717873621","heartbeat_back_addr":"172.21.15.118:6825/1717873621","heartbeat_front_addr":"172.21.15.118:6823/1717873621","state":["exists","up"]},{"osd":3,"uuid":"1c7254e0-d2fd-4b11-a110-04f9d7aa04a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":1195800036},{"type":"v1","addr":"172.21.15.118:6827","nonce":1195800036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":1195800036},{"type":"v1","addr":"172.21.15.118:6829","nonce":1195800036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1195800036},{"type":"v1","addr":"172.21.15.118:6833","nonce":1195800036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":1195800036},{"type":"v1","addr":"172.21.15.118:6831","nonce":1195800036}]},"public_addr":"172.21.15.118:6827/1195800036","cluster_addr":"172.21.15.118:6829/1195800036","heartbeat_back_addr":"172.21.15.118:6833/1195800036","heartbeat_front_addr":"172.21.15.118:6831/1195800036","state":["exists","up"]},{"osd":4,"uuid":"46dcae68-591d-41d4-9829-ce25368caffa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1673072938},{"type":"v1","addr":"172.21.15.170:6801","nonce":1673072938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1673072938},{"type":"v1","addr":"172.21.15.170:6803","nonce":1673072938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1673072938},{"type":"v1","addr":"172.21.15.170:6807","nonce":1673072938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1673072938},{"type":"v1","addr":"172.21.15.170:6805","nonce":1673072938}]},"public_addr":"172.21.15.170:6801/1673072938","cluster_addr":"172.21.15.170:6803/1673072938","heartbeat_back_addr":"172.21.15.170:6807/1673072938","heartbeat_front_addr":"172.21.15.170:6805/1673072938","state":["exists","up"]},{"osd":5,"uuid":"d58f1343-ad2a-4153-89f7-e7d691aa5282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3749911609},{"type":"v1","addr":"172.21.15.170:6809","nonce":3749911609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3749911609},{"type":"v1","addr":"172.21.15.170:6811","nonce":3749911609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3749911609},{"type":"v1","addr":"172.21.15.170:6815","nonce":3749911609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3749911609},{"type":"v1","addr":"172.21.15.170:6813","nonce":3749911609}]},"public_addr":"172.21.15.170:6809/3749911609","cluster_addr":"172.21.15.170:6811/3749911609","heartbeat_back_addr":"172.21.15.170:6815/3749911609","heartbeat_front_addr":"172.21.15.170:6813/3749911609","state":["exists","up"]},{"osd":6,"uuid":"7b63464b-4cfc-4225-91a4-ca4fe84adeb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":965836527},{"type":"v1","addr":"172.21.15.170:6817","nonce":965836527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":965836527},{"type":"v1","addr":"172.21.15.170:6819","nonce":965836527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":965836527},{"type":"v1","addr":"172.21.15.170:6823","nonce":965836527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":965836527},{"type":"v1","addr":"172.21.15.170:6821","nonce":965836527}]},"public_addr":"172.21.15.170:6817/965836527","cluster_addr":"172.21.15.170:6819/965836527","heartbeat_back_addr":"172.21.15.170:6823/965836527","heartbeat_front_addr":"172.21.15.170:6821/965836527","state":["exists","up"]},{"osd":7,"uuid":"3431c0a5-527d-4199-b0e4-c17fc99f8981","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3980860619},{"type":"v1","addr":"172.21.15.170:6825","nonce":3980860619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3980860619},{"type":"v1","addr":"172.21.15.170:6827","nonce":3980860619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3980860619},{"type":"v1","addr":"172.21.15.170:6831","nonce":3980860619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3980860619},{"type":"v1","addr":"172.21.15.170:6829","nonce":3980860619}]},"public_addr":"172.21.15.170:6825/3980860619","cluster_addr":"172.21.15.170:6827/3980860619","heartbeat_back_addr":"172.21.15.170:6831/3980860619","heartbeat_front_addr":"172.21.15.170:6829/3980860619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:29:00.821543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:29:25.706259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:29:51.063464+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:30:17.959114+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:30:40.030347+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:31:03.713849+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:31:27.429096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:31:51.600524+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.118:0/4056359646":"2024-09-18T08:32:44.038764+0000","172.21.15.118:0/3471266426":"2024-09-18T08:32:44.038764+0000","172.21.15.118:0/677224718":"2024-09-18T08:27:13.370329+0000","172.21.15.118:0/3611715606":"2024-09-18T08:27:13.370329+0000","172.21.15.118:0/1555118856":"2024-09-18T08:32:44.038764+0000","172.21.15.118:6800/862659270":"2024-09-18T08:27:13.370329+0000","172.21.15.118:6801/2556545142":"2024-09-18T08:32:44.038764+0000","172.21.15.118:6801/862659270":"2024-09-18T08:27:13.370329+0000","172.21.15.118:6800/2223255332":"2024-09-18T08:27:37.329435+0000","172.21.15.118:6801/2223255332":"2024-09-18T08:27:37.329435+0000","172.21.15.118:0/1111468500":"2024-09-18T08:27:37.329435+0000","172.21.15.118:6800/2556545142":"2024-09-18T08:32:44.038764+0000","172.21.15.118:0/2512398536":"2024-09-18T08:27:37.329435+0000","172.21.15.118:0/858276273":"2024-09-18T08:27:13.370329+0000","172.21.15.118:0/1798368975":"2024-09-18T08:27:37.329435+0000","172.21.15.118:0/1111095991":"2024-09-18T08:32:44.038764+0000","172.21.15.118:0/345247749":"2024-09-18T08:32:44.038764+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T08:32:48.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[41418]: [17/Sep/2024:08:32:46] ENGINE Bus STARTING 2024-09-17T08:32:48.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[41418]: [17/Sep/2024:08:32:46] ENGINE Serving on http://172.21.15.118:8765 2024-09-17T08:32:48.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[41418]: [17/Sep/2024:08:32:46] ENGINE Serving on https://172.21.15.118:7150 2024-09-17T08:32:48.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[41418]: [17/Sep/2024:08:32:46] ENGINE Bus STARTED 2024-09-17T08:32:48.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[41418]: [17/Sep/2024:08:32:46] ENGINE Error in HTTPServer.serve 2024-09-17T08:32:48.278 INFO:journalctl@ceph.mon.c.smithi118.stdout: Traceback (most recent call last): 2024-09-17T08:32:48.278 INFO:journalctl@ceph.mon.c.smithi118.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T08:32:48.278 INFO:journalctl@ceph.mon.c.smithi118.stdout: self._connections.run(self.expiration_interval) 2024-09-17T08:32:48.278 INFO:journalctl@ceph.mon.c.smithi118.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T08:32:48.278 INFO:journalctl@ceph.mon.c.smithi118.stdout: self._run(expiration_interval) 2024-09-17T08:32:48.278 INFO:journalctl@ceph.mon.c.smithi118.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T08:32:48.278 INFO:journalctl@ceph.mon.c.smithi118.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T08:32:48.278 INFO:journalctl@ceph.mon.c.smithi118.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T08:32:48.278 INFO:journalctl@ceph.mon.c.smithi118.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T08:32:48.278 INFO:journalctl@ceph.mon.c.smithi118.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T08:32:48.278 INFO:journalctl@ceph.mon.c.smithi118.stdout: s = self.context.wrap_socket( 2024-09-17T08:32:48.278 INFO:journalctl@ceph.mon.c.smithi118.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T08:32:48.278 INFO:journalctl@ceph.mon.c.smithi118.stdout: return self.sslsocket_class._create( 2024-09-17T08:32:48.279 INFO:journalctl@ceph.mon.c.smithi118.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T08:32:48.279 INFO:journalctl@ceph.mon.c.smithi118.stdout: self.do_handshake() 2024-09-17T08:32:48.279 INFO:journalctl@ceph.mon.c.smithi118.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T08:32:48.279 INFO:journalctl@ceph.mon.c.smithi118.stdout: self._sslobj.do_handshake() 2024-09-17T08:32:48.279 INFO:journalctl@ceph.mon.c.smithi118.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T08:32:48.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:48.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:48.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:48.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:48.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:48.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:48.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:48.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:48.280 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:48.280 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:48.280 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:48.280 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:48.280 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:48.280 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:48.280 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:48.281 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[29491]: [17/Sep/2024:08:32:46] ENGINE Bus STARTING 2024-09-17T08:32:48.281 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[29491]: [17/Sep/2024:08:32:46] ENGINE Serving on http://172.21.15.118:8765 2024-09-17T08:32:48.281 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[29491]: [17/Sep/2024:08:32:46] ENGINE Serving on https://172.21.15.118:7150 2024-09-17T08:32:48.281 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[29491]: [17/Sep/2024:08:32:46] ENGINE Bus STARTED 2024-09-17T08:32:48.281 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[29491]: [17/Sep/2024:08:32:46] ENGINE Error in HTTPServer.serve 2024-09-17T08:32:48.281 INFO:journalctl@ceph.mon.a.smithi118.stdout: Traceback (most recent call last): 2024-09-17T08:32:48.281 INFO:journalctl@ceph.mon.a.smithi118.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T08:32:48.281 INFO:journalctl@ceph.mon.a.smithi118.stdout: self._connections.run(self.expiration_interval) 2024-09-17T08:32:48.281 INFO:journalctl@ceph.mon.a.smithi118.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T08:32:48.281 INFO:journalctl@ceph.mon.a.smithi118.stdout: self._run(expiration_interval) 2024-09-17T08:32:48.281 INFO:journalctl@ceph.mon.a.smithi118.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T08:32:48.281 INFO:journalctl@ceph.mon.a.smithi118.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T08:32:48.281 INFO:journalctl@ceph.mon.a.smithi118.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T08:32:48.282 INFO:journalctl@ceph.mon.a.smithi118.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T08:32:48.282 INFO:journalctl@ceph.mon.a.smithi118.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T08:32:48.282 INFO:journalctl@ceph.mon.a.smithi118.stdout: s = self.context.wrap_socket( 2024-09-17T08:32:48.282 INFO:journalctl@ceph.mon.a.smithi118.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T08:32:48.282 INFO:journalctl@ceph.mon.a.smithi118.stdout: return self.sslsocket_class._create( 2024-09-17T08:32:48.282 INFO:journalctl@ceph.mon.a.smithi118.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T08:32:48.282 INFO:journalctl@ceph.mon.a.smithi118.stdout: self.do_handshake() 2024-09-17T08:32:48.282 INFO:journalctl@ceph.mon.a.smithi118.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T08:32:48.282 INFO:journalctl@ceph.mon.a.smithi118.stdout: self._sslobj.do_handshake() 2024-09-17T08:32:48.282 INFO:journalctl@ceph.mon.a.smithi118.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T08:32:48.282 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:48.282 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:48.282 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:48.282 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:48.283 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:48.283 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:48.283 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:48.283 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:48.283 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:48.283 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:48.283 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:48.283 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:48.283 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:48.283 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:48.283 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:48 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:48.395 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:32:48 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:32:48.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:48 smithi170 ceph-mon[33271]: [17/Sep/2024:08:32:46] ENGINE Bus STARTING 2024-09-17T08:32:48.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:48 smithi170 ceph-mon[33271]: [17/Sep/2024:08:32:46] ENGINE Serving on http://172.21.15.118:8765 2024-09-17T08:32:48.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:48 smithi170 ceph-mon[33271]: [17/Sep/2024:08:32:46] ENGINE Serving on https://172.21.15.118:7150 2024-09-17T08:32:48.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:48 smithi170 ceph-mon[33271]: [17/Sep/2024:08:32:46] ENGINE Bus STARTED 2024-09-17T08:32:48.397 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:48 smithi170 ceph-mon[33271]: [17/Sep/2024:08:32:46] ENGINE Error in HTTPServer.serve 2024-09-17T08:32:48.397 INFO:journalctl@ceph.mon.b.smithi170.stdout: Traceback (most recent call last): 2024-09-17T08:32:48.397 INFO:journalctl@ceph.mon.b.smithi170.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T08:32:48.397 INFO:journalctl@ceph.mon.b.smithi170.stdout: self._connections.run(self.expiration_interval) 2024-09-17T08:32:48.397 INFO:journalctl@ceph.mon.b.smithi170.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T08:32:48.397 INFO:journalctl@ceph.mon.b.smithi170.stdout: self._run(expiration_interval) 2024-09-17T08:32:48.398 INFO:journalctl@ceph.mon.b.smithi170.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T08:32:48.398 INFO:journalctl@ceph.mon.b.smithi170.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T08:32:48.398 INFO:journalctl@ceph.mon.b.smithi170.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T08:32:48.398 INFO:journalctl@ceph.mon.b.smithi170.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T08:32:48.398 INFO:journalctl@ceph.mon.b.smithi170.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T08:32:48.398 INFO:journalctl@ceph.mon.b.smithi170.stdout: s = self.context.wrap_socket( 2024-09-17T08:32:48.399 INFO:journalctl@ceph.mon.b.smithi170.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T08:32:48.399 INFO:journalctl@ceph.mon.b.smithi170.stdout: return self.sslsocket_class._create( 2024-09-17T08:32:48.399 INFO:journalctl@ceph.mon.b.smithi170.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T08:32:48.399 INFO:journalctl@ceph.mon.b.smithi170.stdout: self.do_handshake() 2024-09-17T08:32:48.399 INFO:journalctl@ceph.mon.b.smithi170.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T08:32:48.399 INFO:journalctl@ceph.mon.b.smithi170.stdout: self._sslobj.do_handshake() 2024-09-17T08:32:48.399 INFO:journalctl@ceph.mon.b.smithi170.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T08:32:48.400 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:48 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:48.400 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:48 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:48.400 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:48 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:48.400 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:48 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:48.400 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:48 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:48.400 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:48 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:48.400 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:48 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:48.401 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:48 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:48.401 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:48 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:48.401 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:48 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:48.401 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:48 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:48.401 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:48 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:48.401 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:48 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:48.402 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:48 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:48.402 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:48 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:49.097 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-17T08:32:49.097 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-17T08:32:49.098 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-17T08:32:49.098 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-17T08:32:49.099 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-17T08:32:49.099 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-17T08:32:49.100 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-17T08:32:49.100 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-17T08:32:49.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:49 smithi170 ceph-mon[33271]: Adjusting osd_memory_target on smithi170 to 3760M 2024-09-17T08:32:49.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:49 smithi170 ceph-mon[33271]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:32:49.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:49 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/2215443295' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T08:32:49.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:49 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:32:49.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:49 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:49.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:49 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:49.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:49 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:49.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:49 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:49.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:49 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:49.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:49 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:49.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:49 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:49.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:49 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:49.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:49 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:49.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:49 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:49.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:49 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:49.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:49 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:32:49.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:49 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:32:49.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:49 smithi170 ceph-mon[33271]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-17T08:32:49.453 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:32:49.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:49 smithi118 ceph-mon[41418]: Adjusting osd_memory_target on smithi170 to 3760M 2024-09-17T08:32:49.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:49 smithi118 ceph-mon[41418]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:32:49.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:49 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/2215443295' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T08:32:49.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:49 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:32:49.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:49 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:49.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:49 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:49.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:49 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:49.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:49 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:49.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:49 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:49.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:49 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:49.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:49 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:49.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:49 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:49.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:49 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:49.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:49 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:49.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:49 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:49.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:49 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:32:49.529 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:49 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:32:49.529 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:49 smithi118 ceph-mon[41418]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-17T08:32:49.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:49 smithi118 ceph-mon[29491]: Adjusting osd_memory_target on smithi170 to 3760M 2024-09-17T08:32:49.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:49 smithi118 ceph-mon[29491]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:32:49.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:49 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/2215443295' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T08:32:49.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:49 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:32:49.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:49 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:49.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:49 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:49.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:49 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:49.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:49 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:49.530 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:49 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:49.530 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:49 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:49.530 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:49 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:49.530 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:49 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:49.530 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:49 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:49.530 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:49 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:32:49.530 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:49 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:49.530 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:49 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:32:49.532 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:49 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:32:49.532 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:49 smithi118 ceph-mon[29491]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-17T08:32:49.543 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:32:49.543 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:32:49.544 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:32:49.545 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:32:49.550 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:32:49.551 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:32:49.878 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:32:50.156 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:50 smithi118 ceph-mon[29491]: Adjusting osd_memory_target on smithi118 to 3760M 2024-09-17T08:32:50.157 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:50 smithi118 ceph-mon[29491]: Updating smithi118:/etc/ceph/ceph.conf 2024-09-17T08:32:50.157 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:50 smithi118 ceph-mon[29491]: Updating smithi170:/etc/ceph/ceph.conf 2024-09-17T08:32:50.157 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:50 smithi118 ceph-mon[29491]: Updating smithi118:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:32:50.157 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:50 smithi118 ceph-mon[29491]: Updating smithi170:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:32:50.157 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:50 smithi118 ceph-mon[29491]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:32:50.157 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:50 smithi118 ceph-mon[29491]: Updating smithi118:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:32:50.157 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:50 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:50.157 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:50 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:50.158 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:50 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:50.158 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:50 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:50.158 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:50 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:50.158 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:50 smithi118 ceph-mon[41418]: Adjusting osd_memory_target on smithi118 to 3760M 2024-09-17T08:32:50.158 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:50 smithi118 ceph-mon[41418]: Updating smithi118:/etc/ceph/ceph.conf 2024-09-17T08:32:50.159 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:50 smithi118 ceph-mon[41418]: Updating smithi170:/etc/ceph/ceph.conf 2024-09-17T08:32:50.159 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:50 smithi118 ceph-mon[41418]: Updating smithi118:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:32:50.159 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:50 smithi118 ceph-mon[41418]: Updating smithi170:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:32:50.159 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:50 smithi118 ceph-mon[41418]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:32:50.159 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:50 smithi118 ceph-mon[41418]: Updating smithi118:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:32:50.159 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:50 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:50.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:50 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:50.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:50 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:50.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:50 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:50.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:50 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:50.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:50 smithi170 ceph-mon[33271]: Adjusting osd_memory_target on smithi118 to 3760M 2024-09-17T08:32:50.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:50 smithi170 ceph-mon[33271]: Updating smithi118:/etc/ceph/ceph.conf 2024-09-17T08:32:50.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:50 smithi170 ceph-mon[33271]: Updating smithi170:/etc/ceph/ceph.conf 2024-09-17T08:32:50.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:50 smithi170 ceph-mon[33271]: Updating smithi118:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:32:50.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:50 smithi170 ceph-mon[33271]: Updating smithi170:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:32:50.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:50 smithi170 ceph-mon[33271]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:32:50.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:50 smithi170 ceph-mon[33271]: Updating smithi118:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:32:50.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:50 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:50.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:50 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:50.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:50 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:50.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:50 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:50.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:50 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:32:51.133 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:51 smithi118 ceph-mon[29491]: Updating smithi170:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:32:51.133 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:51 smithi118 ceph-mon[29491]: Updating smithi118:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:32:51.134 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:51 smithi118 ceph-mon[29491]: Deploying daemon alertmanager.a on smithi118 2024-09-17T08:32:51.134 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:51 smithi118 ceph-mon[29491]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:32:51.134 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:51 smithi118 ceph-mon[41418]: Updating smithi170:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:32:51.134 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:51 smithi118 ceph-mon[41418]: Updating smithi118:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:32:51.134 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:51 smithi118 ceph-mon[41418]: Deploying daemon alertmanager.a on smithi118 2024-09-17T08:32:51.134 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:51 smithi118 ceph-mon[41418]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:32:51.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:51 smithi170 ceph-mon[33271]: Updating smithi170:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:32:51.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:51 smithi170 ceph-mon[33271]: Updating smithi118:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:32:51.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:51 smithi170 ceph-mon[33271]: Deploying daemon alertmanager.a on smithi118 2024-09-17T08:32:51.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:51 smithi170 ceph-mon[33271]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:32:53.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:53 smithi118 ceph-mon[29491]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-09-17T08:32:53.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:53 smithi118 ceph-mon[41418]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-09-17T08:32:53.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:53 smithi170 ceph-mon[33271]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-09-17T08:32:55.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:55 smithi118 ceph-mon[29491]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-09-17T08:32:55.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:55 smithi118 ceph-mon[41418]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-09-17T08:32:55.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:55 smithi170 ceph-mon[33271]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-09-17T08:32:57.249 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:57 smithi118 ceph-mon[29491]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-17T08:32:57.250 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:32:57 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:32:57] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:32:57.250 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:57 smithi118 ceph-mon[41418]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-17T08:32:57.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:57 smithi170 ceph-mon[33271]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-17T08:32:58.645 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:32:58 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:32:59.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:59 smithi118 ceph-mon[29491]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-17T08:32:59.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:32:59 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:32:59.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:59 smithi118 ceph-mon[41418]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-17T08:32:59.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:32:59 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:32:59.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:59 smithi170 ceph-mon[33271]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-17T08:32:59.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:32:59 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:00.393 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:00 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:33:00.393 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:00 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:33:00.394 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:00 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:33:01.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:01 smithi118 ceph-mon[29491]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-17T08:33:01.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:01 smithi118 ceph-mon[41418]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-17T08:33:01.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:01 smithi170 ceph-mon[33271]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-17T08:33:03.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:03 smithi118 ceph-mon[29491]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-17T08:33:03.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:03 smithi118 ceph-mon[41418]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-17T08:33:03.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:03 smithi170 ceph-mon[33271]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-17T08:33:05.174 INFO:teuthology.orchestra.run.smithi118.stdout:77309411368 2024-09-17T08:33:05.174 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-17T08:33:05.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:05 smithi170 ceph-mon[33271]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:05.397 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:05 smithi118 ceph-mon[29491]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:05.398 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:05 smithi118 ceph-mon[41418]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:06.366 INFO:teuthology.orchestra.run.smithi118.stdout:163208757275 2024-09-17T08:33:06.366 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-17T08:33:07.078 INFO:teuthology.orchestra.run.smithi118.stdout:197568495637 2024-09-17T08:33:07.078 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-17T08:33:07.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:07 smithi170 ceph-mon[33271]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:07.485 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:07 smithi118 ceph-mon[29491]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:07.486 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:33:07 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:33:07] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:33:07.486 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:07 smithi118 ceph-mon[41418]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:07.505 INFO:teuthology.orchestra.run.smithi118.stdout:55834574895 2024-09-17T08:33:07.505 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-17T08:33:08.645 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:33:08 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:33:09.081 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:09 smithi118 ceph-mon[41418]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:09.082 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:09 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:09.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:09 smithi170 ceph-mon[33271]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:09.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:09 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:09.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:09 smithi118 ceph-mon[29491]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:09.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:09 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:09.937 INFO:teuthology.orchestra.run.smithi118.stdout:128849018911 2024-09-17T08:33:09.937 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-17T08:33:10.567 INFO:teuthology.orchestra.run.smithi118.stdout:34359738418 2024-09-17T08:33:10.567 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-17T08:33:11.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:11 smithi170 ceph-mon[33271]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:11.475 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:11 smithi118 ceph-mon[29491]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:11.476 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:11 smithi118 ceph-mon[41418]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:12.069 INFO:teuthology.orchestra.run.smithi118.stdout:107374182434 2024-09-17T08:33:12.070 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-17T08:33:12.086 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:12 smithi118 systemd[1]: Starting Ceph alertmanager.a for 7862d192-74ce-11ef-bceb-c7b262605968... 2024-09-17T08:33:12.572 INFO:teuthology.orchestra.run.smithi118.stdout:231928234001 2024-09-17T08:33:12.572 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-17T08:33:12.607 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:12.609 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:12.610 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:12.624 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:12.655 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:13.230 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:13 smithi118 ceph-mon[29491]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:13.231 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:13 smithi118 ceph-mon[41418]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:13.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:13 smithi170 ceph-mon[33271]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:13.926 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:13.963 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:15.378 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:15 smithi118 ceph-mon[29491]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:15.378 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:15 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:33:15.378 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:15 smithi118 ceph-mon[41418]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:15.378 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:15 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:33:15.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:15 smithi170 ceph-mon[33271]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:15.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:15 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:33:17.271 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:17 smithi118 ceph-mon[29491]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:17.272 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:33:17 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:33:17] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:33:17.272 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:17 smithi118 ceph-mon[41418]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:17.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:17 smithi170 ceph-mon[33271]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:18.645 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:33:18 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:33:19.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:19 smithi170 ceph-mon[33271]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:19.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:19 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:19.444 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:19 smithi118 ceph-mon[29491]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:19.444 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:19 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:19.444 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:19 smithi118 ceph-mon[41418]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:19.444 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:19 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:20.980 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:20 smithi118 podman[76013]: 2024-09-17 08:33:20.637423341 +0000 UTC m=+1.478424469 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-09-17T08:33:21.260 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:21 smithi118 ceph-mon[29491]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:21.260 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:21 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/2301217426' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-17T08:33:21.260 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:21 smithi118 ceph-mon[41418]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:21.260 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:21 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/2301217426' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-17T08:33:21.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:21 smithi170 ceph-mon[33271]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:21.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:21 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/2301217426' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-17T08:33:21.436 INFO:teuthology.orchestra.run.smithi118.stdout:77309411372 2024-09-17T08:33:21.527 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:21 smithi118 podman[76013]: 2024-09-17 08:33:21.36126426 +0000 UTC m=+2.202265383 volume create 5e09c1896cec06655e2ccb6726454e904150453407d43eeb9e6147d1b5bc6f27 2024-09-17T08:33:21.527 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:21 smithi118 podman[76013]: 2024-09-17 08:33:21.486957133 +0000 UTC m=+2.327958246 container create fa53fcaa2dfa341d54eb14bd967b307405a6f6241d0ce3b001c4d268880431ce (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T08:33:21.805 INFO:teuthology.orchestra.run.smithi118.stdout:107374182438 2024-09-17T08:33:22.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:22 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/166261743' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-17T08:33:22.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:22 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/166261743' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-17T08:33:22.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:22 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/166261743' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-17T08:33:22.591 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182434 got 107374182438 for osd.3 2024-09-17T08:33:22.592 DEBUG:teuthology.parallel:result is None 2024-09-17T08:33:23.210 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:23 smithi118 ceph-mon[29491]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:23.210 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:23 smithi118 ceph-mon[41418]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:23.232 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411368 got 77309411372 for osd.2 2024-09-17T08:33:23.233 DEBUG:teuthology.parallel:result is None 2024-09-17T08:33:23.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:23 smithi170 ceph-mon[33271]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:23.958 INFO:teuthology.orchestra.run.smithi118.stdout:197568495641 2024-09-17T08:33:24.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:24 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3527446310' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-17T08:33:24.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:24 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/3527446310' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-17T08:33:24.277 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:24 smithi118 podman[76013]: 2024-09-17 08:33:24.042678489 +0000 UTC m=+4.883679601 container init fa53fcaa2dfa341d54eb14bd967b307405a6f6241d0ce3b001c4d268880431ce (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T08:33:24.277 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:24 smithi118 podman[76013]: 2024-09-17 08:33:24.04781168 +0000 UTC m=+4.888812796 container start fa53fcaa2dfa341d54eb14bd967b307405a6f6241d0ce3b001c4d268880431ce (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T08:33:24.277 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:24 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a[76364]: ts=2024-09-17T08:33:24.061Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-17T08:33:24.278 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:24 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a[76364]: ts=2024-09-17T08:33:24.061Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-17T08:33:24.278 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:24 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a[76364]: ts=2024-09-17T08:33:24.062Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.118 port=9094 2024-09-17T08:33:24.278 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:24 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a[76364]: ts=2024-09-17T08:33:24.063Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-17T08:33:24.278 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:24 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a[76364]: ts=2024-09-17T08:33:24.086Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-17T08:33:24.278 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:24 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a[76364]: ts=2024-09-17T08:33:24.086Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-17T08:33:24.278 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:24 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a[76364]: ts=2024-09-17T08:33:24.089Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-17T08:33:24.278 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:24 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a[76364]: ts=2024-09-17T08:33:24.089Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-17T08:33:24.278 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:24 smithi118 bash[76013]: fa53fcaa2dfa341d54eb14bd967b307405a6f6241d0ce3b001c4d268880431ce 2024-09-17T08:33:24.278 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:24 smithi118 systemd[1]: Started Ceph alertmanager.a for 7862d192-74ce-11ef-bceb-c7b262605968. 2024-09-17T08:33:24.394 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:24 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/3527446310' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-17T08:33:24.527 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:33:24 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: [17/Sep/2024:08:33:24] ENGINE Bus STOPPING 2024-09-17T08:33:24.853 INFO:teuthology.orchestra.run.smithi118.stdout:128849018915 2024-09-17T08:33:24.853 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:33:24 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: [17/Sep/2024:08:33:24] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-17T08:33:24.853 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:33:24 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: [17/Sep/2024:08:33:24] ENGINE Bus STOPPED 2024-09-17T08:33:24.853 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:33:24 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: [17/Sep/2024:08:33:24] ENGINE Bus STARTING 2024-09-17T08:33:24.853 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:33:24 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: [17/Sep/2024:08:33:24] ENGINE Serving on http://:::9283 2024-09-17T08:33:24.853 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:33:24 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: [17/Sep/2024:08:33:24] ENGINE Bus STARTED 2024-09-17T08:33:25.115 INFO:teuthology.orchestra.run.smithi118.stdout:34359738422 2024-09-17T08:33:25.314 INFO:teuthology.orchestra.run.smithi118.stdout:55834574898 2024-09-17T08:33:25.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:25 smithi118 ceph-mon[41418]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:25.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:25 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:25.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:25 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:25.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:25 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:25.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:25 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:25.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:25 smithi118 ceph-mon[41418]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-17T08:33:25.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:25 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:25.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:25 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:25.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:25 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-17T08:33:25.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:25 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:25.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:25 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/1520319093' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-17T08:33:25.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:25 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/1067459109' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-17T08:33:25.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:25 smithi118 ceph-mon[29491]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:25.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:25 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:25.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:25 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:25.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:25 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:25.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:25 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:25.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:25 smithi118 ceph-mon[29491]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-17T08:33:25.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:25 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:25.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:25 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:25.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:25 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-17T08:33:25.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:25 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:25.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:25 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/1520319093' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-17T08:33:25.530 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:25 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/1067459109' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-17T08:33:25.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:25 smithi170 ceph-mon[33271]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:25.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:25 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:25.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:25 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:25.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:25 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:25.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:25 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:25.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:25 smithi170 ceph-mon[33271]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-17T08:33:25.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:25 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:25.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:25 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:25.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:25 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-17T08:33:25.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:25 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:25.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:25 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/1520319093' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-17T08:33:25.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:25 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/1067459109' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-17T08:33:26.150 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:26 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a[76364]: ts=2024-09-17T08:33:26.063Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.00014323s 2024-09-17T08:33:26.173 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738418 got 34359738422 for osd.0 2024-09-17T08:33:26.173 DEBUG:teuthology.parallel:result is None 2024-09-17T08:33:26.409 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:26 smithi118 ceph-mon[29491]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-17T08:33:26.409 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:26 smithi118 ceph-mon[29491]: Deploying daemon grafana.a on smithi170 2024-09-17T08:33:26.409 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:26 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3880089581' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-17T08:33:26.409 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:26 smithi118 ceph-mon[41418]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-17T08:33:26.409 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:26 smithi118 ceph-mon[41418]: Deploying daemon grafana.a on smithi170 2024-09-17T08:33:26.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:26 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/3880089581' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-17T08:33:26.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:26 smithi170 ceph-mon[33271]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-17T08:33:26.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:26 smithi170 ceph-mon[33271]: Deploying daemon grafana.a on smithi170 2024-09-17T08:33:26.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:26 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/3880089581' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-17T08:33:26.921 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574895 got 55834574898 for osd.1 2024-09-17T08:33:26.921 DEBUG:teuthology.parallel:result is None 2024-09-17T08:33:27.520 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:27 smithi118 ceph-mon[29491]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:27.520 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:27 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3506982230' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-17T08:33:27.521 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:33:27 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:33:27] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:33:27.521 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:27 smithi118 ceph-mon[41418]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:27.521 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:27 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/3506982230' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-17T08:33:27.541 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495637 got 197568495641 for osd.6 2024-09-17T08:33:27.541 DEBUG:teuthology.parallel:result is None 2024-09-17T08:33:27.644 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:27 smithi170 ceph-mon[33271]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:27.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:27 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/3506982230' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-17T08:33:27.653 INFO:teuthology.orchestra.run.smithi118.stdout:163208757279 2024-09-17T08:33:28.317 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757275 got 163208757279 for osd.5 2024-09-17T08:33:28.317 DEBUG:teuthology.parallel:result is None 2024-09-17T08:33:28.484 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:28.645 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:33:28 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:33:28.938 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018911 got 128849018915 for osd.4 2024-09-17T08:33:28.938 DEBUG:teuthology.parallel:result is None 2024-09-17T08:33:29.433 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:29 smithi118 ceph-mon[29491]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:29.433 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:29 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:29.433 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:29 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:33:29.433 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:29 smithi118 ceph-mon[41418]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:29.433 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:29 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:29.433 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:29 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:33:29.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:29 smithi170 ceph-mon[33271]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:29.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:29 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:29.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:29 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:33:30.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:30 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:30.487 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:30 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:30.487 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:30 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:30.890 INFO:teuthology.orchestra.run.smithi118.stdout:231928234006 2024-09-17T08:33:31.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:31 smithi118 ceph-mon[29491]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:31.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:31 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/71966772' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-17T08:33:31.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:31 smithi118 ceph-mon[41418]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:31.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:31 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/71966772' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-17T08:33:31.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:31 smithi170 ceph-mon[33271]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:31.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:31 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/71966772' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-17T08:33:32.034 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234001 got 231928234006 for osd.7 2024-09-17T08:33:32.035 DEBUG:teuthology.parallel:result is None 2024-09-17T08:33:32.035 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-17T08:33:32.035 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-17T08:33:32.250 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:33.502 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:33 smithi118 ceph-mon[29491]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:33.502 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:33 smithi118 ceph-mon[41418]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:33.506 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:33 smithi170 ceph-mon[33271]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:34.233 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:33:34.234 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:34 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a[76364]: ts=2024-09-17T08:33:34.064Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.001459071s 2024-09-17T08:33:34.240 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-09-17T08:33:34.896 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-17T08:33:34.063170+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596246,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":999,"num_read_kb":845,"num_write":609,"num_write_kb":2459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":1771682,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":598,"ondisk_log_size":598,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":232296,"kb_used_data":7952,"kb_used_omap":12,"kb_used_meta":224243,"kb_avail":749728920,"statfs":{"total":767960285184,"available":767722414080,"internally_reserved":0,"allocated":8142848,"data_stored":4430937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":229625401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002905"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.440752+0000","last_change":"2024-09-17T08:32:06.517049+0000","last_active":"2024-09-17T08:32:38.440752+0000","last_peered":"2024-09-17T08:32:38.440752+0000","last_clean":"2024-09-17T08:32:38.440752+0000","last_became_active":"2024-09-17T08:32:06.516602+0000","last_became_peered":"2024-09-17T08:32:06.516602+0000","last_unstale":"2024-09-17T08:32:38.440752+0000","last_undegraded":"2024-09-17T08:32:38.440752+0000","last_fullsized":"2024-09-17T08:32:38.440752+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:33:39.991581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"67'9","reported_seq":35,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443204+0000","last_change":"2024-09-17T08:32:08.530330+0000","last_active":"2024-09-17T08:32:38.443204+0000","last_peered":"2024-09-17T08:32:38.443204+0000","last_clean":"2024-09-17T08:32:38.443204+0000","last_became_active":"2024-09-17T08:32:08.530083+0000","last_became_peered":"2024-09-17T08:32:08.530083+0000","last_unstale":"2024-09-17T08:32:38.443204+0000","last_undegraded":"2024-09-17T08:32:38.443204+0000","last_fullsized":"2024-09-17T08:32:38.443204+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:29:13.722157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.444550+0000","last_change":"2024-09-17T08:32:10.538280+0000","last_active":"2024-09-17T08:32:38.444550+0000","last_peered":"2024-09-17T08:32:38.444550+0000","last_clean":"2024-09-17T08:32:38.444550+0000","last_became_active":"2024-09-17T08:32:10.537573+0000","last_became_peered":"2024-09-17T08:32:10.537573+0000","last_unstale":"2024-09-17T08:32:38.444550+0000","last_undegraded":"2024-09-17T08:32:38.444550+0000","last_fullsized":"2024-09-17T08:32:38.444550+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:35:24.010164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443333+0000","last_change":"2024-09-17T08:32:12.543966+0000","last_active":"2024-09-17T08:32:38.443333+0000","last_peered":"2024-09-17T08:32:38.443333+0000","last_clean":"2024-09-17T08:32:38.443333+0000","last_became_active":"2024-09-17T08:32:12.543687+0000","last_became_peered":"2024-09-17T08:32:12.543687+0000","last_unstale":"2024-09-17T08:32:38.443333+0000","last_undegraded":"2024-09-17T08:32:38.443333+0000","last_fullsized":"2024-09-17T08:32:38.443333+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:23:01.598906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.440694+0000","last_change":"2024-09-17T08:32:12.694907+0000","last_active":"2024-09-17T08:32:38.440694+0000","last_peered":"2024-09-17T08:32:38.440694+0000","last_clean":"2024-09-17T08:32:38.440694+0000","last_became_active":"2024-09-17T08:32:12.694607+0000","last_became_peered":"2024-09-17T08:32:12.694607+0000","last_unstale":"2024-09-17T08:32:38.440694+0000","last_undegraded":"2024-09-17T08:32:38.440694+0000","last_fullsized":"2024-09-17T08:32:38.440694+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:34.532157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.440476+0000","last_change":"2024-09-17T08:32:06.519591+0000","last_active":"2024-09-17T08:32:38.440476+0000","last_peered":"2024-09-17T08:32:38.440476+0000","last_clean":"2024-09-17T08:32:38.440476+0000","last_became_active":"2024-09-17T08:32:06.519301+0000","last_became_peered":"2024-09-17T08:32:06.519301+0000","last_unstale":"2024-09-17T08:32:38.440476+0000","last_undegraded":"2024-09-17T08:32:38.440476+0000","last_fullsized":"2024-09-17T08:32:38.440476+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:18.417256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"67'15","reported_seq":44,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441441+0000","last_change":"2024-09-17T08:32:08.524076+0000","last_active":"2024-09-17T08:32:38.441441+0000","last_peered":"2024-09-17T08:32:38.441441+0000","last_clean":"2024-09-17T08:32:38.441441+0000","last_became_active":"2024-09-17T08:32:08.523778+0000","last_became_peered":"2024-09-17T08:32:08.523778+0000","last_unstale":"2024-09-17T08:32:38.441441+0000","last_undegraded":"2024-09-17T08:32:38.441441+0000","last_fullsized":"2024-09-17T08:32:38.441441+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:49:14.449660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443174+0000","last_change":"2024-09-17T08:32:10.543440+0000","last_active":"2024-09-17T08:32:38.443174+0000","last_peered":"2024-09-17T08:32:38.443174+0000","last_clean":"2024-09-17T08:32:38.443174+0000","last_became_active":"2024-09-17T08:32:10.542455+0000","last_became_peered":"2024-09-17T08:32:10.542455+0000","last_unstale":"2024-09-17T08:32:38.443174+0000","last_undegraded":"2024-09-17T08:32:38.443174+0000","last_fullsized":"2024-09-17T08:32:38.443174+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:27:52.637574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441912+0000","last_change":"2024-09-17T08:32:06.514230+0000","last_active":"2024-09-17T08:32:38.441912+0000","last_peered":"2024-09-17T08:32:38.441912+0000","last_clean":"2024-09-17T08:32:38.441912+0000","last_became_active":"2024-09-17T08:32:06.513655+0000","last_became_peered":"2024-09-17T08:32:06.513655+0000","last_unstale":"2024-09-17T08:32:38.441912+0000","last_undegraded":"2024-09-17T08:32:38.441912+0000","last_fullsized":"2024-09-17T08:32:38.441912+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:35:35.109613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"67'9","reported_seq":35,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443589+0000","last_change":"2024-09-17T08:32:08.533286+0000","last_active":"2024-09-17T08:32:38.443589+0000","last_peered":"2024-09-17T08:32:38.443589+0000","last_clean":"2024-09-17T08:32:38.443589+0000","last_became_active":"2024-09-17T08:32:08.533023+0000","last_became_peered":"2024-09-17T08:32:08.533023+0000","last_unstale":"2024-09-17T08:32:38.443589+0000","last_undegraded":"2024-09-17T08:32:38.443589+0000","last_fullsized":"2024-09-17T08:32:38.443589+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:56:39.845687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441532+0000","last_change":"2024-09-17T08:32:10.541683+0000","last_active":"2024-09-17T08:32:38.441532+0000","last_peered":"2024-09-17T08:32:38.441532+0000","last_clean":"2024-09-17T08:32:38.441532+0000","last_became_active":"2024-09-17T08:32:10.541279+0000","last_became_peered":"2024-09-17T08:32:10.541279+0000","last_unstale":"2024-09-17T08:32:38.441532+0000","last_undegraded":"2024-09-17T08:32:38.441532+0000","last_fullsized":"2024-09-17T08:32:38.441532+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:11:16.225567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439371+0000","last_change":"2024-09-17T08:32:12.548178+0000","last_active":"2024-09-17T08:32:38.439371+0000","last_peered":"2024-09-17T08:32:38.439371+0000","last_clean":"2024-09-17T08:32:38.439371+0000","last_became_active":"2024-09-17T08:32:12.546998+0000","last_became_peered":"2024-09-17T08:32:12.546998+0000","last_unstale":"2024-09-17T08:32:38.439371+0000","last_undegraded":"2024-09-17T08:32:38.439371+0000","last_fullsized":"2024-09-17T08:32:38.439371+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:40:38.518257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.442167+0000","last_change":"2024-09-17T08:32:06.517632+0000","last_active":"2024-09-17T08:32:38.442167+0000","last_peered":"2024-09-17T08:32:38.442167+0000","last_clean":"2024-09-17T08:32:38.442167+0000","last_became_active":"2024-09-17T08:32:06.517334+0000","last_became_peered":"2024-09-17T08:32:06.517334+0000","last_unstale":"2024-09-17T08:32:38.442167+0000","last_undegraded":"2024-09-17T08:32:38.442167+0000","last_fullsized":"2024-09-17T08:32:38.442167+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:50:35.513812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"67'5","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443428+0000","last_change":"2024-09-17T08:32:08.532942+0000","last_active":"2024-09-17T08:32:38.443428+0000","last_peered":"2024-09-17T08:32:38.443428+0000","last_clean":"2024-09-17T08:32:38.443428+0000","last_became_active":"2024-09-17T08:32:08.532650+0000","last_became_peered":"2024-09-17T08:32:08.532650+0000","last_unstale":"2024-09-17T08:32:38.443428+0000","last_undegraded":"2024-09-17T08:32:38.443428+0000","last_fullsized":"2024-09-17T08:32:38.443428+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:48:47.558738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439344+0000","last_change":"2024-09-17T08:32:10.543302+0000","last_active":"2024-09-17T08:32:38.439344+0000","last_peered":"2024-09-17T08:32:38.439344+0000","last_clean":"2024-09-17T08:32:38.439344+0000","last_became_active":"2024-09-17T08:32:10.543029+0000","last_became_peered":"2024-09-17T08:32:10.543029+0000","last_unstale":"2024-09-17T08:32:38.439344+0000","last_undegraded":"2024-09-17T08:32:38.439344+0000","last_fullsized":"2024-09-17T08:32:38.439344+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:50:51.167211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441449+0000","last_change":"2024-09-17T08:32:12.544708+0000","last_active":"2024-09-17T08:32:38.441449+0000","last_peered":"2024-09-17T08:32:38.441449+0000","last_clean":"2024-09-17T08:32:38.441449+0000","last_became_active":"2024-09-17T08:32:12.544384+0000","last_became_peered":"2024-09-17T08:32:12.544384+0000","last_unstale":"2024-09-17T08:32:38.441449+0000","last_undegraded":"2024-09-17T08:32:38.441449+0000","last_fullsized":"2024-09-17T08:32:38.441449+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:50:27.675858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443915+0000","last_change":"2024-09-17T08:32:12.694656+0000","last_active":"2024-09-17T08:32:38.443915+0000","last_peered":"2024-09-17T08:32:38.443915+0000","last_clean":"2024-09-17T08:32:38.443915+0000","last_became_active":"2024-09-17T08:32:12.694380+0000","last_became_peered":"2024-09-17T08:32:12.694380+0000","last_unstale":"2024-09-17T08:32:38.443915+0000","last_undegraded":"2024-09-17T08:32:38.443915+0000","last_fullsized":"2024-09-17T08:32:38.443915+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:29:59.529947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.440713+0000","last_change":"2024-09-17T08:32:06.516203+0000","last_active":"2024-09-17T08:32:38.440713+0000","last_peered":"2024-09-17T08:32:38.440713+0000","last_clean":"2024-09-17T08:32:38.440713+0000","last_became_active":"2024-09-17T08:32:06.515783+0000","last_became_peered":"2024-09-17T08:32:06.515783+0000","last_unstale":"2024-09-17T08:32:38.440713+0000","last_undegraded":"2024-09-17T08:32:38.440713+0000","last_fullsized":"2024-09-17T08:32:38.440713+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:16:53.826036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"67'15","reported_seq":44,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.444825+0000","last_change":"2024-09-17T08:32:08.528844+0000","last_active":"2024-09-17T08:32:38.444825+0000","last_peered":"2024-09-17T08:32:38.444825+0000","last_clean":"2024-09-17T08:32:38.444825+0000","last_became_active":"2024-09-17T08:32:08.528562+0000","last_became_peered":"2024-09-17T08:32:08.528562+0000","last_unstale":"2024-09-17T08:32:38.444825+0000","last_undegraded":"2024-09-17T08:32:38.444825+0000","last_fullsized":"2024-09-17T08:32:38.444825+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:25:25.572514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.444476+0000","last_change":"2024-09-17T08:32:10.546518+0000","last_active":"2024-09-17T08:32:38.444476+0000","last_peered":"2024-09-17T08:32:38.444476+0000","last_clean":"2024-09-17T08:32:38.444476+0000","last_became_active":"2024-09-17T08:32:10.546204+0000","last_became_peered":"2024-09-17T08:32:10.546204+0000","last_unstale":"2024-09-17T08:32:38.444476+0000","last_undegraded":"2024-09-17T08:32:38.444476+0000","last_fullsized":"2024-09-17T08:32:38.444476+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:32:51.438201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441069+0000","last_change":"2024-09-17T08:32:12.694565+0000","last_active":"2024-09-17T08:32:38.441069+0000","last_peered":"2024-09-17T08:32:38.441069+0000","last_clean":"2024-09-17T08:32:38.441069+0000","last_became_active":"2024-09-17T08:32:12.694235+0000","last_became_peered":"2024-09-17T08:32:12.694235+0000","last_unstale":"2024-09-17T08:32:38.441069+0000","last_undegraded":"2024-09-17T08:32:38.441069+0000","last_fullsized":"2024-09-17T08:32:38.441069+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:30:07.758747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443769+0000","last_change":"2024-09-17T08:32:06.515516+0000","last_active":"2024-09-17T08:32:38.443769+0000","last_peered":"2024-09-17T08:32:38.443769+0000","last_clean":"2024-09-17T08:32:38.443769+0000","last_became_active":"2024-09-17T08:32:06.515246+0000","last_became_peered":"2024-09-17T08:32:06.515246+0000","last_unstale":"2024-09-17T08:32:38.443769+0000","last_undegraded":"2024-09-17T08:32:38.443769+0000","last_fullsized":"2024-09-17T08:32:38.443769+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:40.273522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"67'12","reported_seq":42,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.440045+0000","last_change":"2024-09-17T08:32:08.528795+0000","last_active":"2024-09-17T08:32:38.440045+0000","last_peered":"2024-09-17T08:32:38.440045+0000","last_clean":"2024-09-17T08:32:38.440045+0000","last_became_active":"2024-09-17T08:32:08.528471+0000","last_became_peered":"2024-09-17T08:32:08.528471+0000","last_unstale":"2024-09-17T08:32:38.440045+0000","last_undegraded":"2024-09-17T08:32:38.440045+0000","last_fullsized":"2024-09-17T08:32:38.440045+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:55.620995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443600+0000","last_change":"2024-09-17T08:32:10.536262+0000","last_active":"2024-09-17T08:32:38.443600+0000","last_peered":"2024-09-17T08:32:38.443600+0000","last_clean":"2024-09-17T08:32:38.443600+0000","last_became_active":"2024-09-17T08:32:10.535942+0000","last_became_peered":"2024-09-17T08:32:10.535942+0000","last_unstale":"2024-09-17T08:32:38.443600+0000","last_undegraded":"2024-09-17T08:32:38.443600+0000","last_fullsized":"2024-09-17T08:32:38.443600+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:10:52.244174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"67'1","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439530+0000","last_change":"2024-09-17T08:32:12.550172+0000","last_active":"2024-09-17T08:32:38.439530+0000","last_peered":"2024-09-17T08:32:38.439530+0000","last_clean":"2024-09-17T08:32:38.439530+0000","last_became_active":"2024-09-17T08:32:12.549901+0000","last_became_peered":"2024-09-17T08:32:12.549901+0000","last_unstale":"2024-09-17T08:32:38.439530+0000","last_undegraded":"2024-09-17T08:32:38.439530+0000","last_fullsized":"2024-09-17T08:32:38.439530+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:03:14.243693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.444733+0000","last_change":"2024-09-17T08:32:06.523880+0000","last_active":"2024-09-17T08:32:38.444733+0000","last_peered":"2024-09-17T08:32:38.444733+0000","last_clean":"2024-09-17T08:32:38.444733+0000","last_became_active":"2024-09-17T08:32:06.523220+0000","last_became_peered":"2024-09-17T08:32:06.523220+0000","last_unstale":"2024-09-17T08:32:38.444733+0000","last_undegraded":"2024-09-17T08:32:38.444733+0000","last_fullsized":"2024-09-17T08:32:38.444733+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:51:58.865000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"67'10","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441017+0000","last_change":"2024-09-17T08:32:08.531794+0000","last_active":"2024-09-17T08:32:38.441017+0000","last_peered":"2024-09-17T08:32:38.441017+0000","last_clean":"2024-09-17T08:32:38.441017+0000","last_became_active":"2024-09-17T08:32:08.531427+0000","last_became_peered":"2024-09-17T08:32:08.531427+0000","last_unstale":"2024-09-17T08:32:38.441017+0000","last_undegraded":"2024-09-17T08:32:38.441017+0000","last_fullsized":"2024-09-17T08:32:38.441017+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:56.671609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"67'8","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T08:33:24.922695+0000","last_change":"2024-09-17T08:32:10.539394+0000","last_active":"2024-09-17T08:33:24.922695+0000","last_peered":"2024-09-17T08:33:24.922695+0000","last_clean":"2024-09-17T08:33:24.922695+0000","last_became_active":"2024-09-17T08:32:10.539094+0000","last_became_peered":"2024-09-17T08:32:10.539094+0000","last_unstale":"2024-09-17T08:33:24.922695+0000","last_undegraded":"2024-09-17T08:33:24.922695+0000","last_fullsized":"2024-09-17T08:33:24.922695+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:59:30.658834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"67'15","reported_seq":44,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.446042+0000","last_change":"2024-09-17T08:32:08.533283+0000","last_active":"2024-09-17T08:32:38.446042+0000","last_peered":"2024-09-17T08:32:38.446042+0000","last_clean":"2024-09-17T08:32:38.446042+0000","last_became_active":"2024-09-17T08:32:08.532988+0000","last_became_peered":"2024-09-17T08:32:08.532988+0000","last_unstale":"2024-09-17T08:32:38.446042+0000","last_undegraded":"2024-09-17T08:32:38.446042+0000","last_fullsized":"2024-09-17T08:32:38.446042+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:28.483744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"60'1","reported_seq":26,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.440107+0000","last_change":"2024-09-17T08:32:06.519832+0000","last_active":"2024-09-17T08:32:38.440107+0000","last_peered":"2024-09-17T08:32:38.440107+0000","last_clean":"2024-09-17T08:32:38.440107+0000","last_became_active":"2024-09-17T08:32:06.519439+0000","last_became_peered":"2024-09-17T08:32:06.519439+0000","last_unstale":"2024-09-17T08:32:38.440107+0000","last_undegraded":"2024-09-17T08:32:38.440107+0000","last_fullsized":"2024-09-17T08:32:38.440107+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:20:54.535189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"67'8","reported_seq":42,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T08:33:24.922670+0000","last_change":"2024-09-17T08:32:10.543873+0000","last_active":"2024-09-17T08:33:24.922670+0000","last_peered":"2024-09-17T08:33:24.922670+0000","last_clean":"2024-09-17T08:33:24.922670+0000","last_became_active":"2024-09-17T08:32:10.543549+0000","last_became_peered":"2024-09-17T08:32:10.543549+0000","last_unstale":"2024-09-17T08:33:24.922670+0000","last_undegraded":"2024-09-17T08:33:24.922670+0000","last_fullsized":"2024-09-17T08:33:24.922670+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:27:40.672094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441601+0000","last_change":"2024-09-17T08:32:12.545361+0000","last_active":"2024-09-17T08:32:38.441601+0000","last_peered":"2024-09-17T08:32:38.441601+0000","last_clean":"2024-09-17T08:32:38.441601+0000","last_became_active":"2024-09-17T08:32:12.545077+0000","last_became_peered":"2024-09-17T08:32:12.545077+0000","last_unstale":"2024-09-17T08:32:38.441601+0000","last_undegraded":"2024-09-17T08:32:38.441601+0000","last_fullsized":"2024-09-17T08:32:38.441601+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:59.216859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"67'18","reported_seq":51,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439516+0000","last_change":"2024-09-17T08:32:08.522483+0000","last_active":"2024-09-17T08:32:38.439516+0000","last_peered":"2024-09-17T08:32:38.439516+0000","last_clean":"2024-09-17T08:32:38.439516+0000","last_became_active":"2024-09-17T08:32:08.522031+0000","last_became_peered":"2024-09-17T08:32:08.522031+0000","last_unstale":"2024-09-17T08:32:38.439516+0000","last_undegraded":"2024-09-17T08:32:38.439516+0000","last_fullsized":"2024-09-17T08:32:38.439516+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:37:58.206984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.440998+0000","last_change":"2024-09-17T08:32:06.518756+0000","last_active":"2024-09-17T08:32:38.440998+0000","last_peered":"2024-09-17T08:32:38.440998+0000","last_clean":"2024-09-17T08:32:38.440998+0000","last_became_active":"2024-09-17T08:32:06.518508+0000","last_became_peered":"2024-09-17T08:32:06.518508+0000","last_unstale":"2024-09-17T08:32:38.440998+0000","last_undegraded":"2024-09-17T08:32:38.440998+0000","last_fullsized":"2024-09-17T08:32:38.440998+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:50:54.216970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443253+0000","last_change":"2024-09-17T08:32:10.539361+0000","last_active":"2024-09-17T08:32:38.443253+0000","last_peered":"2024-09-17T08:32:38.443253+0000","last_clean":"2024-09-17T08:32:38.443253+0000","last_became_active":"2024-09-17T08:32:10.536390+0000","last_became_peered":"2024-09-17T08:32:10.536390+0000","last_unstale":"2024-09-17T08:32:38.443253+0000","last_undegraded":"2024-09-17T08:32:38.443253+0000","last_fullsized":"2024-09-17T08:32:38.443253+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:32:56.028375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443444+0000","last_change":"2024-09-17T08:32:12.549946+0000","last_active":"2024-09-17T08:32:38.443444+0000","last_peered":"2024-09-17T08:32:38.443444+0000","last_clean":"2024-09-17T08:32:38.443444+0000","last_became_active":"2024-09-17T08:32:12.548943+0000","last_became_peered":"2024-09-17T08:32:12.548943+0000","last_unstale":"2024-09-17T08:32:38.443444+0000","last_undegraded":"2024-09-17T08:32:38.443444+0000","last_fullsized":"2024-09-17T08:32:38.443444+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:47:00.317734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"67'14","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443393+0000","last_change":"2024-09-17T08:32:08.527353+0000","last_active":"2024-09-17T08:32:38.443393+0000","last_peered":"2024-09-17T08:32:38.443393+0000","last_clean":"2024-09-17T08:32:38.443393+0000","last_became_active":"2024-09-17T08:32:08.527074+0000","last_became_peered":"2024-09-17T08:32:08.527074+0000","last_unstale":"2024-09-17T08:32:38.443393+0000","last_undegraded":"2024-09-17T08:32:38.443393+0000","last_fullsized":"2024-09-17T08:32:38.443393+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:08.200483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441160+0000","last_change":"2024-09-17T08:32:06.515084+0000","last_active":"2024-09-17T08:32:38.441160+0000","last_peered":"2024-09-17T08:32:38.441160+0000","last_clean":"2024-09-17T08:32:38.441160+0000","last_became_active":"2024-09-17T08:32:06.514813+0000","last_became_peered":"2024-09-17T08:32:06.514813+0000","last_unstale":"2024-09-17T08:32:38.441160+0000","last_undegraded":"2024-09-17T08:32:38.441160+0000","last_fullsized":"2024-09-17T08:32:38.441160+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:29:11.823668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"67'8","reported_seq":42,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T08:33:24.922483+0000","last_change":"2024-09-17T08:32:10.535432+0000","last_active":"2024-09-17T08:33:24.922483+0000","last_peered":"2024-09-17T08:33:24.922483+0000","last_clean":"2024-09-17T08:33:24.922483+0000","last_became_active":"2024-09-17T08:32:10.535175+0000","last_became_peered":"2024-09-17T08:32:10.535175+0000","last_unstale":"2024-09-17T08:33:24.922483+0000","last_undegraded":"2024-09-17T08:33:24.922483+0000","last_fullsized":"2024-09-17T08:33:24.922483+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:42:15.467257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439696+0000","last_change":"2024-09-17T08:32:12.696753+0000","last_active":"2024-09-17T08:32:38.439696+0000","last_peered":"2024-09-17T08:32:38.439696+0000","last_clean":"2024-09-17T08:32:38.439696+0000","last_became_active":"2024-09-17T08:32:12.696495+0000","last_became_peered":"2024-09-17T08:32:12.696495+0000","last_unstale":"2024-09-17T08:32:38.439696+0000","last_undegraded":"2024-09-17T08:32:38.439696+0000","last_fullsized":"2024-09-17T08:32:38.439696+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:39:49.719496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"67'10","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.444401+0000","last_change":"2024-09-17T08:32:08.527446+0000","last_active":"2024-09-17T08:32:38.444401+0000","last_peered":"2024-09-17T08:32:38.444401+0000","last_clean":"2024-09-17T08:32:38.444401+0000","last_became_active":"2024-09-17T08:32:08.527129+0000","last_became_peered":"2024-09-17T08:32:08.527129+0000","last_unstale":"2024-09-17T08:32:38.444401+0000","last_undegraded":"2024-09-17T08:32:38.444401+0000","last_fullsized":"2024-09-17T08:32:38.444401+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:02:25.088242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441928+0000","last_change":"2024-09-17T08:32:06.520832+0000","last_active":"2024-09-17T08:32:38.441928+0000","last_peered":"2024-09-17T08:32:38.441928+0000","last_clean":"2024-09-17T08:32:38.441928+0000","last_became_active":"2024-09-17T08:32:06.520547+0000","last_became_peered":"2024-09-17T08:32:06.520547+0000","last_unstale":"2024-09-17T08:32:38.441928+0000","last_undegraded":"2024-09-17T08:32:38.441928+0000","last_fullsized":"2024-09-17T08:32:38.441928+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:07.463011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"67'8","reported_seq":42,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T08:33:24.922425+0000","last_change":"2024-09-17T08:32:10.543123+0000","last_active":"2024-09-17T08:33:24.922425+0000","last_peered":"2024-09-17T08:33:24.922425+0000","last_clean":"2024-09-17T08:33:24.922425+0000","last_became_active":"2024-09-17T08:32:10.542768+0000","last_became_peered":"2024-09-17T08:32:10.542768+0000","last_unstale":"2024-09-17T08:33:24.922425+0000","last_undegraded":"2024-09-17T08:33:24.922425+0000","last_fullsized":"2024-09-17T08:33:24.922425+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:11:46.728776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439470+0000","last_change":"2024-09-17T08:32:12.548678+0000","last_active":"2024-09-17T08:32:38.439470+0000","last_peered":"2024-09-17T08:32:38.439470+0000","last_clean":"2024-09-17T08:32:38.439470+0000","last_became_active":"2024-09-17T08:32:12.548412+0000","last_became_peered":"2024-09-17T08:32:12.548412+0000","last_unstale":"2024-09-17T08:32:38.439470+0000","last_undegraded":"2024-09-17T08:32:38.439470+0000","last_fullsized":"2024-09-17T08:32:38.439470+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:51.271230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"67'19","reported_seq":55,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.440453+0000","last_change":"2024-09-17T08:32:08.525359+0000","last_active":"2024-09-17T08:32:38.440453+0000","last_peered":"2024-09-17T08:32:38.440453+0000","last_clean":"2024-09-17T08:32:38.440453+0000","last_became_active":"2024-09-17T08:32:08.525089+0000","last_became_peered":"2024-09-17T08:32:08.525089+0000","last_unstale":"2024-09-17T08:32:38.440453+0000","last_undegraded":"2024-09-17T08:32:38.440453+0000","last_fullsized":"2024-09-17T08:32:38.440453+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:09.002769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.445662+0000","last_change":"2024-09-17T08:32:06.517321+0000","last_active":"2024-09-17T08:32:38.445662+0000","last_peered":"2024-09-17T08:32:38.445662+0000","last_clean":"2024-09-17T08:32:38.445662+0000","last_became_active":"2024-09-17T08:32:06.517061+0000","last_became_peered":"2024-09-17T08:32:06.517061+0000","last_unstale":"2024-09-17T08:32:38.445662+0000","last_undegraded":"2024-09-17T08:32:38.445662+0000","last_fullsized":"2024-09-17T08:32:38.445662+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:32:00.392092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.446466+0000","last_change":"2024-09-17T08:32:10.539083+0000","last_active":"2024-09-17T08:32:38.446466+0000","last_peered":"2024-09-17T08:32:38.446466+0000","last_clean":"2024-09-17T08:32:38.446466+0000","last_became_active":"2024-09-17T08:32:10.538761+0000","last_became_peered":"2024-09-17T08:32:10.538761+0000","last_unstale":"2024-09-17T08:32:38.446466+0000","last_undegraded":"2024-09-17T08:32:38.446466+0000","last_fullsized":"2024-09-17T08:32:38.446466+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:43:03.304139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.444423+0000","last_change":"2024-09-17T08:32:12.694826+0000","last_active":"2024-09-17T08:32:38.444423+0000","last_peered":"2024-09-17T08:32:38.444423+0000","last_clean":"2024-09-17T08:32:38.444423+0000","last_became_active":"2024-09-17T08:32:12.694275+0000","last_became_peered":"2024-09-17T08:32:12.694275+0000","last_unstale":"2024-09-17T08:32:38.444423+0000","last_undegraded":"2024-09-17T08:32:38.444423+0000","last_fullsized":"2024-09-17T08:32:38.444423+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:25:45.341443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"70'29","reported_seq":46,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T08:33:24.922875+0000","last_change":"2024-09-17T08:32:38.460715+0000","last_active":"2024-09-17T08:33:24.922875+0000","last_peered":"2024-09-17T08:33:24.922875+0000","last_clean":"2024-09-17T08:33:24.922875+0000","last_became_active":"2024-09-17T08:32:38.446928+0000","last_became_peered":"2024-09-17T08:32:38.446928+0000","last_unstale":"2024-09-17T08:33:24.922875+0000","last_undegraded":"2024-09-17T08:33:24.922875+0000","last_fullsized":"2024-09-17T08:33:24.922875+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:08:15.827269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443705+0000","last_change":"2024-09-17T08:32:06.514090+0000","last_active":"2024-09-17T08:32:38.443705+0000","last_peered":"2024-09-17T08:32:38.443705+0000","last_clean":"2024-09-17T08:32:38.443705+0000","last_became_active":"2024-09-17T08:32:06.513811+0000","last_became_peered":"2024-09-17T08:32:06.513811+0000","last_unstale":"2024-09-17T08:32:38.443705+0000","last_undegraded":"2024-09-17T08:32:38.443705+0000","last_fullsized":"2024-09-17T08:32:38.443705+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:24:06.372273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"64'2","reported_seq":28,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441528+0000","last_change":"2024-09-17T08:32:10.534037+0000","last_active":"2024-09-17T08:32:38.441528+0000","last_peered":"2024-09-17T08:32:38.441528+0000","last_clean":"2024-09-17T08:32:38.441528+0000","last_became_active":"2024-09-17T08:32:06.514988+0000","last_became_peered":"2024-09-17T08:32:06.514988+0000","last_unstale":"2024-09-17T08:32:38.441528+0000","last_undegraded":"2024-09-17T08:32:38.441528+0000","last_fullsized":"2024-09-17T08:32:38.441528+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:46.008515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020955869999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.440399+0000","last_change":"2024-09-17T08:32:10.546500+0000","last_active":"2024-09-17T08:32:38.440399+0000","last_peered":"2024-09-17T08:32:38.440399+0000","last_clean":"2024-09-17T08:32:38.440399+0000","last_became_active":"2024-09-17T08:32:10.546079+0000","last_became_peered":"2024-09-17T08:32:10.546079+0000","last_unstale":"2024-09-17T08:32:38.440399+0000","last_undegraded":"2024-09-17T08:32:38.440399+0000","last_fullsized":"2024-09-17T08:32:38.440399+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:43:17.414063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439728+0000","last_change":"2024-09-17T08:32:12.546332+0000","last_active":"2024-09-17T08:32:38.439728+0000","last_peered":"2024-09-17T08:32:38.439728+0000","last_clean":"2024-09-17T08:32:38.439728+0000","last_became_active":"2024-09-17T08:32:12.545993+0000","last_became_peered":"2024-09-17T08:32:12.545993+0000","last_unstale":"2024-09-17T08:32:38.439728+0000","last_undegraded":"2024-09-17T08:32:38.439728+0000","last_fullsized":"2024-09-17T08:32:38.439728+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:11:00.528909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"67'13","reported_seq":46,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.446141+0000","last_change":"2024-09-17T08:32:08.527426+0000","last_active":"2024-09-17T08:32:38.446141+0000","last_peered":"2024-09-17T08:32:38.446141+0000","last_clean":"2024-09-17T08:32:38.446141+0000","last_became_active":"2024-09-17T08:32:08.527030+0000","last_became_peered":"2024-09-17T08:32:08.527030+0000","last_unstale":"2024-09-17T08:32:38.446141+0000","last_undegraded":"2024-09-17T08:32:38.446141+0000","last_fullsized":"2024-09-17T08:32:38.446141+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:06:54.129541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"60'1","reported_seq":25,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.446676+0000","last_change":"2024-09-17T08:32:06.515617+0000","last_active":"2024-09-17T08:32:38.446676+0000","last_peered":"2024-09-17T08:32:38.446676+0000","last_clean":"2024-09-17T08:32:38.446676+0000","last_became_active":"2024-09-17T08:32:06.515325+0000","last_became_peered":"2024-09-17T08:32:06.515325+0000","last_unstale":"2024-09-17T08:32:38.446676+0000","last_undegraded":"2024-09-17T08:32:38.446676+0000","last_fullsized":"2024-09-17T08:32:38.446676+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:01:22.001817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"62'1","reported_seq":27,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443735+0000","last_change":"2024-09-17T08:32:10.528937+0000","last_active":"2024-09-17T08:32:38.443735+0000","last_peered":"2024-09-17T08:32:38.443735+0000","last_clean":"2024-09-17T08:32:38.443735+0000","last_became_active":"2024-09-17T08:32:06.512453+0000","last_became_peered":"2024-09-17T08:32:06.512453+0000","last_unstale":"2024-09-17T08:32:38.443735+0000","last_undegraded":"2024-09-17T08:32:38.443735+0000","last_fullsized":"2024-09-17T08:32:38.443735+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:49:16.482240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042636900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.444475+0000","last_change":"2024-09-17T08:32:10.536503+0000","last_active":"2024-09-17T08:32:38.444475+0000","last_peered":"2024-09-17T08:32:38.444475+0000","last_clean":"2024-09-17T08:32:38.444475+0000","last_became_active":"2024-09-17T08:32:10.536201+0000","last_became_peered":"2024-09-17T08:32:10.536201+0000","last_unstale":"2024-09-17T08:32:38.444475+0000","last_undegraded":"2024-09-17T08:32:38.444475+0000","last_fullsized":"2024-09-17T08:32:38.444475+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:35:41.890464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439686+0000","last_change":"2024-09-17T08:32:12.694472+0000","last_active":"2024-09-17T08:32:38.439686+0000","last_peered":"2024-09-17T08:32:38.439686+0000","last_clean":"2024-09-17T08:32:38.439686+0000","last_became_active":"2024-09-17T08:32:12.694188+0000","last_became_peered":"2024-09-17T08:32:12.694188+0000","last_unstale":"2024-09-17T08:32:38.439686+0000","last_undegraded":"2024-09-17T08:32:38.439686+0000","last_fullsized":"2024-09-17T08:32:38.439686+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:48:38.549255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"67'12","reported_seq":37,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441075+0000","last_change":"2024-09-17T08:32:08.526555+0000","last_active":"2024-09-17T08:32:38.441075+0000","last_peered":"2024-09-17T08:32:38.441075+0000","last_clean":"2024-09-17T08:32:38.441075+0000","last_became_active":"2024-09-17T08:32:08.526232+0000","last_became_peered":"2024-09-17T08:32:08.526232+0000","last_unstale":"2024-09-17T08:32:38.441075+0000","last_undegraded":"2024-09-17T08:32:38.441075+0000","last_fullsized":"2024-09-17T08:32:38.441075+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:31:46.911842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439985+0000","last_change":"2024-09-17T08:32:06.519792+0000","last_active":"2024-09-17T08:32:38.439985+0000","last_peered":"2024-09-17T08:32:38.439985+0000","last_clean":"2024-09-17T08:32:38.439985+0000","last_became_active":"2024-09-17T08:32:06.519537+0000","last_became_peered":"2024-09-17T08:32:06.519537+0000","last_unstale":"2024-09-17T08:32:38.439985+0000","last_undegraded":"2024-09-17T08:32:38.439985+0000","last_fullsized":"2024-09-17T08:32:38.439985+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:43:26.888478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"68'5","reported_seq":106,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T08:33:28.281271+0000","last_change":"2024-09-17T08:32:10.532085+0000","last_active":"2024-09-17T08:33:28.281271+0000","last_peered":"2024-09-17T08:33:28.281271+0000","last_clean":"2024-09-17T08:33:28.281271+0000","last_became_active":"2024-09-17T08:32:06.520521+0000","last_became_peered":"2024-09-17T08:32:06.520521+0000","last_unstale":"2024-09-17T08:33:28.281271+0000","last_undegraded":"2024-09-17T08:33:28.281271+0000","last_fullsized":"2024-09-17T08:33:28.281271+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:04:42.051173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000319551,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":77,"num_read_kb":72,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.442067+0000","last_change":"2024-09-17T08:32:10.540716+0000","last_active":"2024-09-17T08:32:38.442067+0000","last_peered":"2024-09-17T08:32:38.442067+0000","last_clean":"2024-09-17T08:32:38.442067+0000","last_became_active":"2024-09-17T08:32:10.540289+0000","last_became_peered":"2024-09-17T08:32:10.540289+0000","last_unstale":"2024-09-17T08:32:38.442067+0000","last_undegraded":"2024-09-17T08:32:38.442067+0000","last_fullsized":"2024-09-17T08:32:38.442067+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:23:35.377736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.445271+0000","last_change":"2024-09-17T08:32:12.550135+0000","last_active":"2024-09-17T08:32:38.445271+0000","last_peered":"2024-09-17T08:32:38.445271+0000","last_clean":"2024-09-17T08:32:38.445271+0000","last_became_active":"2024-09-17T08:32:12.549831+0000","last_became_peered":"2024-09-17T08:32:12.549831+0000","last_unstale":"2024-09-17T08:32:38.445271+0000","last_undegraded":"2024-09-17T08:32:38.445271+0000","last_fullsized":"2024-09-17T08:32:38.445271+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:30:31.912823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"67'14","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.446309+0000","last_change":"2024-09-17T08:32:08.525573+0000","last_active":"2024-09-17T08:32:38.446309+0000","last_peered":"2024-09-17T08:32:38.446309+0000","last_clean":"2024-09-17T08:32:38.446309+0000","last_became_active":"2024-09-17T08:32:08.525277+0000","last_became_peered":"2024-09-17T08:32:08.525277+0000","last_unstale":"2024-09-17T08:32:38.446309+0000","last_undegraded":"2024-09-17T08:32:38.446309+0000","last_fullsized":"2024-09-17T08:32:38.446309+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:39:38.647697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439991+0000","last_change":"2024-09-17T08:32:06.520661+0000","last_active":"2024-09-17T08:32:38.439991+0000","last_peered":"2024-09-17T08:32:38.439991+0000","last_clean":"2024-09-17T08:32:38.439991+0000","last_became_active":"2024-09-17T08:32:06.520414+0000","last_became_peered":"2024-09-17T08:32:06.520414+0000","last_unstale":"2024-09-17T08:32:38.439991+0000","last_undegraded":"2024-09-17T08:32:38.439991+0000","last_fullsized":"2024-09-17T08:32:38.439991+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:47:17.219305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"71'128","reported_seq":266,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T08:32:46.105522+0000","last_change":"2024-09-17T08:31:56.830022+0000","last_active":"2024-09-17T08:32:46.105522+0000","last_peered":"2024-09-17T08:32:46.105522+0000","last_clean":"2024-09-17T08:32:46.105522+0000","last_became_active":"2024-09-17T08:31:56.829586+0000","last_became_peered":"2024-09-17T08:31:56.829586+0000","last_unstale":"2024-09-17T08:32:46.105522+0000","last_undegraded":"2024-09-17T08:32:46.105522+0000","last_fullsized":"2024-09-17T08:32:46.105522+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:29:53.628973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:29:53.628973+0000","last_clean_scrub_stamp":"2024-09-17T08:29:53.628973+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:27:27.657455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439614+0000","last_change":"2024-09-17T08:32:10.537678+0000","last_active":"2024-09-17T08:32:38.439614+0000","last_peered":"2024-09-17T08:32:38.439614+0000","last_clean":"2024-09-17T08:32:38.439614+0000","last_became_active":"2024-09-17T08:32:10.537387+0000","last_became_peered":"2024-09-17T08:32:10.537387+0000","last_unstale":"2024-09-17T08:32:38.439614+0000","last_undegraded":"2024-09-17T08:32:38.439614+0000","last_fullsized":"2024-09-17T08:32:38.439614+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:03:35.883626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.442013+0000","last_change":"2024-09-17T08:32:12.541915+0000","last_active":"2024-09-17T08:32:38.442013+0000","last_peered":"2024-09-17T08:32:38.442013+0000","last_clean":"2024-09-17T08:32:38.442013+0000","last_became_active":"2024-09-17T08:32:12.541620+0000","last_became_peered":"2024-09-17T08:32:12.541620+0000","last_unstale":"2024-09-17T08:32:38.442013+0000","last_undegraded":"2024-09-17T08:32:38.442013+0000","last_fullsized":"2024-09-17T08:32:38.442013+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:42:21.610221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"67'11","reported_seq":59,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.453255+0000","last_change":"2024-09-17T08:32:38.453025+0000","last_active":"2024-09-17T08:32:38.453255+0000","last_peered":"2024-09-17T08:32:38.453255+0000","last_clean":"2024-09-17T08:32:38.453255+0000","last_became_active":"2024-09-17T08:32:38.447402+0000","last_became_peered":"2024-09-17T08:32:38.447402+0000","last_unstale":"2024-09-17T08:32:38.453255+0000","last_undegraded":"2024-09-17T08:32:38.453255+0000","last_fullsized":"2024-09-17T08:32:38.453255+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:44:36.541038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443596+0000","last_change":"2024-09-17T08:32:06.514823+0000","last_active":"2024-09-17T08:32:38.443596+0000","last_peered":"2024-09-17T08:32:38.443596+0000","last_clean":"2024-09-17T08:32:38.443596+0000","last_became_active":"2024-09-17T08:32:06.514130+0000","last_became_peered":"2024-09-17T08:32:06.514130+0000","last_unstale":"2024-09-17T08:32:38.443596+0000","last_undegraded":"2024-09-17T08:32:38.443596+0000","last_fullsized":"2024-09-17T08:32:38.443596+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:25:57.872879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441834+0000","last_change":"2024-09-17T08:32:10.542601+0000","last_active":"2024-09-17T08:32:38.441834+0000","last_peered":"2024-09-17T08:32:38.441834+0000","last_clean":"2024-09-17T08:32:38.441834+0000","last_became_active":"2024-09-17T08:32:10.542295+0000","last_became_peered":"2024-09-17T08:32:10.542295+0000","last_unstale":"2024-09-17T08:32:38.441834+0000","last_undegraded":"2024-09-17T08:32:38.441834+0000","last_fullsized":"2024-09-17T08:32:38.441834+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:06:48.334079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439596+0000","last_change":"2024-09-17T08:32:12.694578+0000","last_active":"2024-09-17T08:32:38.439596+0000","last_peered":"2024-09-17T08:32:38.439596+0000","last_clean":"2024-09-17T08:32:38.439596+0000","last_became_active":"2024-09-17T08:32:12.694275+0000","last_became_peered":"2024-09-17T08:32:12.694275+0000","last_unstale":"2024-09-17T08:32:38.439596+0000","last_undegraded":"2024-09-17T08:32:38.439596+0000","last_fullsized":"2024-09-17T08:32:38.439596+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:57:22.633148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"67'17","reported_seq":74,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.467792+0000","last_change":"2024-09-17T08:32:38.467605+0000","last_active":"2024-09-17T08:32:38.467792+0000","last_peered":"2024-09-17T08:32:38.467792+0000","last_clean":"2024-09-17T08:32:38.467792+0000","last_became_active":"2024-09-17T08:32:38.447388+0000","last_became_peered":"2024-09-17T08:32:38.447388+0000","last_unstale":"2024-09-17T08:32:38.467792+0000","last_undegraded":"2024-09-17T08:32:38.467792+0000","last_fullsized":"2024-09-17T08:32:38.467792+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:16.348652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.446798+0000","last_change":"2024-09-17T08:32:06.515756+0000","last_active":"2024-09-17T08:32:38.446798+0000","last_peered":"2024-09-17T08:32:38.446798+0000","last_clean":"2024-09-17T08:32:38.446798+0000","last_became_active":"2024-09-17T08:32:06.515481+0000","last_became_peered":"2024-09-17T08:32:06.515481+0000","last_unstale":"2024-09-17T08:32:38.446798+0000","last_undegraded":"2024-09-17T08:32:38.446798+0000","last_fullsized":"2024-09-17T08:32:38.446798+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:04:33.289415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.445365+0000","last_change":"2024-09-17T08:32:10.546682+0000","last_active":"2024-09-17T08:32:38.445365+0000","last_peered":"2024-09-17T08:32:38.445365+0000","last_clean":"2024-09-17T08:32:38.445365+0000","last_became_active":"2024-09-17T08:32:10.546132+0000","last_became_peered":"2024-09-17T08:32:10.546132+0000","last_unstale":"2024-09-17T08:32:38.445365+0000","last_undegraded":"2024-09-17T08:32:38.445365+0000","last_fullsized":"2024-09-17T08:32:38.445365+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:28:01.966126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.444691+0000","last_change":"2024-09-17T08:32:12.546672+0000","last_active":"2024-09-17T08:32:38.444691+0000","last_peered":"2024-09-17T08:32:38.444691+0000","last_clean":"2024-09-17T08:32:38.444691+0000","last_became_active":"2024-09-17T08:32:12.546417+0000","last_became_peered":"2024-09-17T08:32:12.546417+0000","last_unstale":"2024-09-17T08:32:38.444691+0000","last_undegraded":"2024-09-17T08:32:38.444691+0000","last_fullsized":"2024-09-17T08:32:38.444691+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:20:23.859665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"67'10","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443674+0000","last_change":"2024-09-17T08:32:08.533942+0000","last_active":"2024-09-17T08:32:38.443674+0000","last_peered":"2024-09-17T08:32:38.443674+0000","last_clean":"2024-09-17T08:32:38.443674+0000","last_became_active":"2024-09-17T08:32:08.533664+0000","last_became_peered":"2024-09-17T08:32:08.533664+0000","last_unstale":"2024-09-17T08:32:38.443674+0000","last_undegraded":"2024-09-17T08:32:38.443674+0000","last_fullsized":"2024-09-17T08:32:38.443674+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:43:57.996920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441013+0000","last_change":"2024-09-17T08:32:06.512628+0000","last_active":"2024-09-17T08:32:38.441013+0000","last_peered":"2024-09-17T08:32:38.441013+0000","last_clean":"2024-09-17T08:32:38.441013+0000","last_became_active":"2024-09-17T08:32:06.512049+0000","last_became_peered":"2024-09-17T08:32:06.512049+0000","last_unstale":"2024-09-17T08:32:38.441013+0000","last_undegraded":"2024-09-17T08:32:38.441013+0000","last_fullsized":"2024-09-17T08:32:38.441013+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:43:37.306297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"67'8","reported_seq":42,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T08:33:24.922425+0000","last_change":"2024-09-17T08:32:10.536769+0000","last_active":"2024-09-17T08:33:24.922425+0000","last_peered":"2024-09-17T08:33:24.922425+0000","last_clean":"2024-09-17T08:33:24.922425+0000","last_became_active":"2024-09-17T08:32:10.536216+0000","last_became_peered":"2024-09-17T08:32:10.536216+0000","last_unstale":"2024-09-17T08:33:24.922425+0000","last_undegraded":"2024-09-17T08:33:24.922425+0000","last_fullsized":"2024-09-17T08:33:24.922425+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:17:57.131915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443764+0000","last_change":"2024-09-17T08:32:12.549748+0000","last_active":"2024-09-17T08:32:38.443764+0000","last_peered":"2024-09-17T08:32:38.443764+0000","last_clean":"2024-09-17T08:32:38.443764+0000","last_became_active":"2024-09-17T08:32:12.549429+0000","last_became_peered":"2024-09-17T08:32:12.549429+0000","last_unstale":"2024-09-17T08:32:38.443764+0000","last_undegraded":"2024-09-17T08:32:38.443764+0000","last_fullsized":"2024-09-17T08:32:38.443764+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:57:32.749218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"67'9","reported_seq":35,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.440367+0000","last_change":"2024-09-17T08:32:08.527343+0000","last_active":"2024-09-17T08:32:38.440367+0000","last_peered":"2024-09-17T08:32:38.440367+0000","last_clean":"2024-09-17T08:32:38.440367+0000","last_became_active":"2024-09-17T08:32:08.527011+0000","last_became_peered":"2024-09-17T08:32:08.527011+0000","last_unstale":"2024-09-17T08:32:38.440367+0000","last_undegraded":"2024-09-17T08:32:38.440367+0000","last_fullsized":"2024-09-17T08:32:38.440367+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:19:36.249578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.442223+0000","last_change":"2024-09-17T08:32:06.521196+0000","last_active":"2024-09-17T08:32:38.442223+0000","last_peered":"2024-09-17T08:32:38.442223+0000","last_clean":"2024-09-17T08:32:38.442223+0000","last_became_active":"2024-09-17T08:32:06.520907+0000","last_became_peered":"2024-09-17T08:32:06.520907+0000","last_unstale":"2024-09-17T08:32:38.442223+0000","last_undegraded":"2024-09-17T08:32:38.442223+0000","last_fullsized":"2024-09-17T08:32:38.442223+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:32:54.848677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443864+0000","last_change":"2024-09-17T08:32:10.537279+0000","last_active":"2024-09-17T08:32:38.443864+0000","last_peered":"2024-09-17T08:32:38.443864+0000","last_clean":"2024-09-17T08:32:38.443864+0000","last_became_active":"2024-09-17T08:32:10.537001+0000","last_became_peered":"2024-09-17T08:32:10.537001+0000","last_unstale":"2024-09-17T08:32:38.443864+0000","last_undegraded":"2024-09-17T08:32:38.443864+0000","last_fullsized":"2024-09-17T08:32:38.443864+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:16:12.797016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439903+0000","last_change":"2024-09-17T08:32:12.549575+0000","last_active":"2024-09-17T08:32:38.439903+0000","last_peered":"2024-09-17T08:32:38.439903+0000","last_clean":"2024-09-17T08:32:38.439903+0000","last_became_active":"2024-09-17T08:32:12.549193+0000","last_became_peered":"2024-09-17T08:32:12.549193+0000","last_unstale":"2024-09-17T08:32:38.439903+0000","last_undegraded":"2024-09-17T08:32:38.439903+0000","last_fullsized":"2024-09-17T08:32:38.439903+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:59:49.773354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"67'19","reported_seq":50,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.446615+0000","last_change":"2024-09-17T08:32:08.528428+0000","last_active":"2024-09-17T08:32:38.446615+0000","last_peered":"2024-09-17T08:32:38.446615+0000","last_clean":"2024-09-17T08:32:38.446615+0000","last_became_active":"2024-09-17T08:32:08.528192+0000","last_became_peered":"2024-09-17T08:32:08.528192+0000","last_unstale":"2024-09-17T08:32:38.446615+0000","last_undegraded":"2024-09-17T08:32:38.446615+0000","last_fullsized":"2024-09-17T08:32:38.446615+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:01.049301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439890+0000","last_change":"2024-09-17T08:32:06.517665+0000","last_active":"2024-09-17T08:32:38.439890+0000","last_peered":"2024-09-17T08:32:38.439890+0000","last_clean":"2024-09-17T08:32:38.439890+0000","last_became_active":"2024-09-17T08:32:06.517407+0000","last_became_peered":"2024-09-17T08:32:06.517407+0000","last_unstale":"2024-09-17T08:32:38.439890+0000","last_undegraded":"2024-09-17T08:32:38.439890+0000","last_fullsized":"2024-09-17T08:32:38.439890+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:58:13.456366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.444072+0000","last_change":"2024-09-17T08:32:10.541218+0000","last_active":"2024-09-17T08:32:38.444072+0000","last_peered":"2024-09-17T08:32:38.444072+0000","last_clean":"2024-09-17T08:32:38.444072+0000","last_became_active":"2024-09-17T08:32:10.540908+0000","last_became_peered":"2024-09-17T08:32:10.540908+0000","last_unstale":"2024-09-17T08:32:38.444072+0000","last_undegraded":"2024-09-17T08:32:38.444072+0000","last_fullsized":"2024-09-17T08:32:38.444072+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:51.618576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439795+0000","last_change":"2024-09-17T08:32:12.550666+0000","last_active":"2024-09-17T08:32:38.439795+0000","last_peered":"2024-09-17T08:32:38.439795+0000","last_clean":"2024-09-17T08:32:38.439795+0000","last_became_active":"2024-09-17T08:32:12.550356+0000","last_became_peered":"2024-09-17T08:32:12.550356+0000","last_unstale":"2024-09-17T08:32:38.439795+0000","last_undegraded":"2024-09-17T08:32:38.439795+0000","last_fullsized":"2024-09-17T08:32:38.439795+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:31:18.051423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"67'12","reported_seq":42,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443489+0000","last_change":"2024-09-17T08:32:08.533629+0000","last_active":"2024-09-17T08:32:38.443489+0000","last_peered":"2024-09-17T08:32:38.443489+0000","last_clean":"2024-09-17T08:32:38.443489+0000","last_became_active":"2024-09-17T08:32:08.533303+0000","last_became_peered":"2024-09-17T08:32:08.533303+0000","last_unstale":"2024-09-17T08:32:38.443489+0000","last_undegraded":"2024-09-17T08:32:38.443489+0000","last_fullsized":"2024-09-17T08:32:38.443489+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:08:05.067425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439968+0000","last_change":"2024-09-17T08:32:06.518413+0000","last_active":"2024-09-17T08:32:38.439968+0000","last_peered":"2024-09-17T08:32:38.439968+0000","last_clean":"2024-09-17T08:32:38.439968+0000","last_became_active":"2024-09-17T08:32:06.518073+0000","last_became_peered":"2024-09-17T08:32:06.518073+0000","last_unstale":"2024-09-17T08:32:38.439968+0000","last_undegraded":"2024-09-17T08:32:38.439968+0000","last_fullsized":"2024-09-17T08:32:38.439968+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:07:08.022690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443416+0000","last_change":"2024-09-17T08:32:10.540363+0000","last_active":"2024-09-17T08:32:38.443416+0000","last_peered":"2024-09-17T08:32:38.443416+0000","last_clean":"2024-09-17T08:32:38.443416+0000","last_became_active":"2024-09-17T08:32:10.540055+0000","last_became_peered":"2024-09-17T08:32:10.540055+0000","last_unstale":"2024-09-17T08:32:38.443416+0000","last_undegraded":"2024-09-17T08:32:38.443416+0000","last_fullsized":"2024-09-17T08:32:38.443416+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:31:37.966877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441143+0000","last_change":"2024-09-17T08:32:12.548919+0000","last_active":"2024-09-17T08:32:38.441143+0000","last_peered":"2024-09-17T08:32:38.441143+0000","last_clean":"2024-09-17T08:32:38.441143+0000","last_became_active":"2024-09-17T08:32:12.546326+0000","last_became_peered":"2024-09-17T08:32:12.546326+0000","last_unstale":"2024-09-17T08:32:38.441143+0000","last_undegraded":"2024-09-17T08:32:38.441143+0000","last_fullsized":"2024-09-17T08:32:38.441143+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:25.361371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"67'15","reported_seq":44,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441318+0000","last_change":"2024-09-17T08:32:08.526395+0000","last_active":"2024-09-17T08:32:38.441318+0000","last_peered":"2024-09-17T08:32:38.441318+0000","last_clean":"2024-09-17T08:32:38.441318+0000","last_became_active":"2024-09-17T08:32:08.526118+0000","last_became_peered":"2024-09-17T08:32:08.526118+0000","last_unstale":"2024-09-17T08:32:38.441318+0000","last_undegraded":"2024-09-17T08:32:38.441318+0000","last_fullsized":"2024-09-17T08:32:38.441318+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:22:45.111550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"60'2","reported_seq":28,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439995+0000","last_change":"2024-09-17T08:32:06.519365+0000","last_active":"2024-09-17T08:32:38.439995+0000","last_peered":"2024-09-17T08:32:38.439995+0000","last_clean":"2024-09-17T08:32:38.439995+0000","last_became_active":"2024-09-17T08:32:06.519106+0000","last_became_peered":"2024-09-17T08:32:06.519106+0000","last_unstale":"2024-09-17T08:32:38.439995+0000","last_undegraded":"2024-09-17T08:32:38.439995+0000","last_fullsized":"2024-09-17T08:32:38.439995+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:15:50.606571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"67'8","reported_seq":42,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T08:33:24.922676+0000","last_change":"2024-09-17T08:32:10.544235+0000","last_active":"2024-09-17T08:33:24.922676+0000","last_peered":"2024-09-17T08:33:24.922676+0000","last_clean":"2024-09-17T08:33:24.922676+0000","last_became_active":"2024-09-17T08:32:10.543872+0000","last_became_peered":"2024-09-17T08:32:10.543872+0000","last_unstale":"2024-09-17T08:33:24.922676+0000","last_undegraded":"2024-09-17T08:33:24.922676+0000","last_fullsized":"2024-09-17T08:33:24.922676+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:47.645731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441372+0000","last_change":"2024-09-17T08:32:12.695219+0000","last_active":"2024-09-17T08:32:38.441372+0000","last_peered":"2024-09-17T08:32:38.441372+0000","last_clean":"2024-09-17T08:32:38.441372+0000","last_became_active":"2024-09-17T08:32:12.694097+0000","last_became_peered":"2024-09-17T08:32:12.694097+0000","last_unstale":"2024-09-17T08:32:38.441372+0000","last_undegraded":"2024-09-17T08:32:38.441372+0000","last_fullsized":"2024-09-17T08:32:38.441372+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:11:20.311913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.445787+0000","last_change":"2024-09-17T08:32:06.516655+0000","last_active":"2024-09-17T08:32:38.445787+0000","last_peered":"2024-09-17T08:32:38.445787+0000","last_clean":"2024-09-17T08:32:38.445787+0000","last_became_active":"2024-09-17T08:32:06.516375+0000","last_became_peered":"2024-09-17T08:32:06.516375+0000","last_unstale":"2024-09-17T08:32:38.445787+0000","last_undegraded":"2024-09-17T08:32:38.445787+0000","last_fullsized":"2024-09-17T08:32:38.445787+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:58:03.022084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"67'6","reported_seq":28,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.440848+0000","last_change":"2024-09-17T08:32:08.532685+0000","last_active":"2024-09-17T08:32:38.440848+0000","last_peered":"2024-09-17T08:32:38.440848+0000","last_clean":"2024-09-17T08:32:38.440848+0000","last_became_active":"2024-09-17T08:32:08.532410+0000","last_became_peered":"2024-09-17T08:32:08.532410+0000","last_unstale":"2024-09-17T08:32:38.440848+0000","last_undegraded":"2024-09-17T08:32:38.440848+0000","last_fullsized":"2024-09-17T08:32:38.440848+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:37:42.638603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441758+0000","last_change":"2024-09-17T08:32:10.537113+0000","last_active":"2024-09-17T08:32:38.441758+0000","last_peered":"2024-09-17T08:32:38.441758+0000","last_clean":"2024-09-17T08:32:38.441758+0000","last_became_active":"2024-09-17T08:32:10.536713+0000","last_became_peered":"2024-09-17T08:32:10.536713+0000","last_unstale":"2024-09-17T08:32:38.441758+0000","last_undegraded":"2024-09-17T08:32:38.441758+0000","last_fullsized":"2024-09-17T08:32:38.441758+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:14.257796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439605+0000","last_change":"2024-09-17T08:32:12.695165+0000","last_active":"2024-09-17T08:32:38.439605+0000","last_peered":"2024-09-17T08:32:38.439605+0000","last_clean":"2024-09-17T08:32:38.439605+0000","last_became_active":"2024-09-17T08:32:12.694913+0000","last_became_peered":"2024-09-17T08:32:12.694913+0000","last_unstale":"2024-09-17T08:32:38.439605+0000","last_undegraded":"2024-09-17T08:32:38.439605+0000","last_fullsized":"2024-09-17T08:32:38.439605+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:11:34.354636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"67'9","reported_seq":35,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441114+0000","last_change":"2024-09-17T08:32:08.523545+0000","last_active":"2024-09-17T08:32:38.441114+0000","last_peered":"2024-09-17T08:32:38.441114+0000","last_clean":"2024-09-17T08:32:38.441114+0000","last_became_active":"2024-09-17T08:32:08.523212+0000","last_became_peered":"2024-09-17T08:32:08.523212+0000","last_unstale":"2024-09-17T08:32:38.441114+0000","last_undegraded":"2024-09-17T08:32:38.441114+0000","last_fullsized":"2024-09-17T08:32:38.441114+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:52:42.189935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439547+0000","last_change":"2024-09-17T08:32:06.518457+0000","last_active":"2024-09-17T08:32:38.439547+0000","last_peered":"2024-09-17T08:32:38.439547+0000","last_clean":"2024-09-17T08:32:38.439547+0000","last_became_active":"2024-09-17T08:32:06.518126+0000","last_became_peered":"2024-09-17T08:32:06.518126+0000","last_unstale":"2024-09-17T08:32:38.439547+0000","last_undegraded":"2024-09-17T08:32:38.439547+0000","last_fullsized":"2024-09-17T08:32:38.439547+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:35:51.082723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.440693+0000","last_change":"2024-09-17T08:32:10.536866+0000","last_active":"2024-09-17T08:32:38.440693+0000","last_peered":"2024-09-17T08:32:38.440693+0000","last_clean":"2024-09-17T08:32:38.440693+0000","last_became_active":"2024-09-17T08:32:10.536586+0000","last_became_peered":"2024-09-17T08:32:10.536586+0000","last_unstale":"2024-09-17T08:32:38.440693+0000","last_undegraded":"2024-09-17T08:32:38.440693+0000","last_fullsized":"2024-09-17T08:32:38.440693+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:20:34.975150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443422+0000","last_change":"2024-09-17T08:32:12.552715+0000","last_active":"2024-09-17T08:32:38.443422+0000","last_peered":"2024-09-17T08:32:38.443422+0000","last_clean":"2024-09-17T08:32:38.443422+0000","last_became_active":"2024-09-17T08:32:12.552430+0000","last_became_peered":"2024-09-17T08:32:12.552430+0000","last_unstale":"2024-09-17T08:32:38.443422+0000","last_undegraded":"2024-09-17T08:32:38.443422+0000","last_fullsized":"2024-09-17T08:32:38.443422+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:43.203832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"67'9","reported_seq":35,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441793+0000","last_change":"2024-09-17T08:32:08.531006+0000","last_active":"2024-09-17T08:32:38.441793+0000","last_peered":"2024-09-17T08:32:38.441793+0000","last_clean":"2024-09-17T08:32:38.441793+0000","last_became_active":"2024-09-17T08:32:08.530746+0000","last_became_peered":"2024-09-17T08:32:08.530746+0000","last_unstale":"2024-09-17T08:32:38.441793+0000","last_undegraded":"2024-09-17T08:32:38.441793+0000","last_fullsized":"2024-09-17T08:32:38.441793+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:11:38.326764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.440052+0000","last_change":"2024-09-17T08:32:06.521875+0000","last_active":"2024-09-17T08:32:38.440052+0000","last_peered":"2024-09-17T08:32:38.440052+0000","last_clean":"2024-09-17T08:32:38.440052+0000","last_became_active":"2024-09-17T08:32:06.521586+0000","last_became_peered":"2024-09-17T08:32:06.521586+0000","last_unstale":"2024-09-17T08:32:38.440052+0000","last_undegraded":"2024-09-17T08:32:38.440052+0000","last_fullsized":"2024-09-17T08:32:38.440052+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:32:15.981385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"67'8","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T08:33:24.922488+0000","last_change":"2024-09-17T08:32:10.533118+0000","last_active":"2024-09-17T08:33:24.922488+0000","last_peered":"2024-09-17T08:33:24.922488+0000","last_clean":"2024-09-17T08:33:24.922488+0000","last_became_active":"2024-09-17T08:32:10.532849+0000","last_became_peered":"2024-09-17T08:32:10.532849+0000","last_unstale":"2024-09-17T08:33:24.922488+0000","last_undegraded":"2024-09-17T08:33:24.922488+0000","last_fullsized":"2024-09-17T08:33:24.922488+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:12:16.625685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443986+0000","last_change":"2024-09-17T08:32:12.549406+0000","last_active":"2024-09-17T08:32:38.443986+0000","last_peered":"2024-09-17T08:32:38.443986+0000","last_clean":"2024-09-17T08:32:38.443986+0000","last_became_active":"2024-09-17T08:32:12.548994+0000","last_became_peered":"2024-09-17T08:32:12.548994+0000","last_unstale":"2024-09-17T08:32:38.443986+0000","last_undegraded":"2024-09-17T08:32:38.443986+0000","last_fullsized":"2024-09-17T08:32:38.443986+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:06.757239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"67'10","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441244+0000","last_change":"2024-09-17T08:32:08.531863+0000","last_active":"2024-09-17T08:32:38.441244+0000","last_peered":"2024-09-17T08:32:38.441244+0000","last_clean":"2024-09-17T08:32:38.441244+0000","last_became_active":"2024-09-17T08:32:08.531586+0000","last_became_peered":"2024-09-17T08:32:08.531586+0000","last_unstale":"2024-09-17T08:32:38.441244+0000","last_undegraded":"2024-09-17T08:32:38.441244+0000","last_fullsized":"2024-09-17T08:32:38.441244+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:41:23.793510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439874+0000","last_change":"2024-09-17T08:32:06.518456+0000","last_active":"2024-09-17T08:32:38.439874+0000","last_peered":"2024-09-17T08:32:38.439874+0000","last_clean":"2024-09-17T08:32:38.439874+0000","last_became_active":"2024-09-17T08:32:06.518097+0000","last_became_peered":"2024-09-17T08:32:06.518097+0000","last_unstale":"2024-09-17T08:32:38.439874+0000","last_undegraded":"2024-09-17T08:32:38.439874+0000","last_fullsized":"2024-09-17T08:32:38.439874+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:06.171179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"67'8","reported_seq":42,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T08:33:24.922672+0000","last_change":"2024-09-17T08:32:10.540578+0000","last_active":"2024-09-17T08:33:24.922672+0000","last_peered":"2024-09-17T08:33:24.922672+0000","last_clean":"2024-09-17T08:33:24.922672+0000","last_became_active":"2024-09-17T08:32:10.540225+0000","last_became_peered":"2024-09-17T08:32:10.540225+0000","last_unstale":"2024-09-17T08:33:24.922672+0000","last_undegraded":"2024-09-17T08:33:24.922672+0000","last_fullsized":"2024-09-17T08:33:24.922672+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:03:26.564210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.440862+0000","last_change":"2024-09-17T08:32:12.548390+0000","last_active":"2024-09-17T08:32:38.440862+0000","last_peered":"2024-09-17T08:32:38.440862+0000","last_clean":"2024-09-17T08:32:38.440862+0000","last_became_active":"2024-09-17T08:32:12.547813+0000","last_became_peered":"2024-09-17T08:32:12.547813+0000","last_unstale":"2024-09-17T08:32:38.440862+0000","last_undegraded":"2024-09-17T08:32:38.440862+0000","last_fullsized":"2024-09-17T08:32:38.440862+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:46.939265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"67'11","reported_seq":38,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443524+0000","last_change":"2024-09-17T08:32:08.530137+0000","last_active":"2024-09-17T08:32:38.443524+0000","last_peered":"2024-09-17T08:32:38.443524+0000","last_clean":"2024-09-17T08:32:38.443524+0000","last_became_active":"2024-09-17T08:32:08.529908+0000","last_became_peered":"2024-09-17T08:32:08.529908+0000","last_unstale":"2024-09-17T08:32:38.443524+0000","last_undegraded":"2024-09-17T08:32:38.443524+0000","last_fullsized":"2024-09-17T08:32:38.443524+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:17.496846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443933+0000","last_change":"2024-09-17T08:32:06.515825+0000","last_active":"2024-09-17T08:32:38.443933+0000","last_peered":"2024-09-17T08:32:38.443933+0000","last_clean":"2024-09-17T08:32:38.443933+0000","last_became_active":"2024-09-17T08:32:06.515261+0000","last_became_peered":"2024-09-17T08:32:06.515261+0000","last_unstale":"2024-09-17T08:32:38.443933+0000","last_undegraded":"2024-09-17T08:32:38.443933+0000","last_fullsized":"2024-09-17T08:32:38.443933+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:33:22.230484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.444216+0000","last_change":"2024-09-17T08:32:10.539278+0000","last_active":"2024-09-17T08:32:38.444216+0000","last_peered":"2024-09-17T08:32:38.444216+0000","last_clean":"2024-09-17T08:32:38.444216+0000","last_became_active":"2024-09-17T08:32:10.538963+0000","last_became_peered":"2024-09-17T08:32:10.538963+0000","last_unstale":"2024-09-17T08:32:38.444216+0000","last_undegraded":"2024-09-17T08:32:38.444216+0000","last_fullsized":"2024-09-17T08:32:38.444216+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:45:25.115543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441188+0000","last_change":"2024-09-17T08:32:12.543598+0000","last_active":"2024-09-17T08:32:38.441188+0000","last_peered":"2024-09-17T08:32:38.441188+0000","last_clean":"2024-09-17T08:32:38.441188+0000","last_became_active":"2024-09-17T08:32:12.543357+0000","last_became_peered":"2024-09-17T08:32:12.543357+0000","last_unstale":"2024-09-17T08:32:38.441188+0000","last_undegraded":"2024-09-17T08:32:38.441188+0000","last_fullsized":"2024-09-17T08:32:38.441188+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:29:40.676098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"67'9","reported_seq":35,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.444560+0000","last_change":"2024-09-17T08:32:08.528603+0000","last_active":"2024-09-17T08:32:38.444560+0000","last_peered":"2024-09-17T08:32:38.444560+0000","last_clean":"2024-09-17T08:32:38.444560+0000","last_became_active":"2024-09-17T08:32:08.528248+0000","last_became_peered":"2024-09-17T08:32:08.528248+0000","last_unstale":"2024-09-17T08:32:38.444560+0000","last_undegraded":"2024-09-17T08:32:38.444560+0000","last_fullsized":"2024-09-17T08:32:38.444560+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:47.118008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439803+0000","last_change":"2024-09-17T08:32:06.523347+0000","last_active":"2024-09-17T08:32:38.439803+0000","last_peered":"2024-09-17T08:32:38.439803+0000","last_clean":"2024-09-17T08:32:38.439803+0000","last_became_active":"2024-09-17T08:32:06.523056+0000","last_became_peered":"2024-09-17T08:32:06.523056+0000","last_unstale":"2024-09-17T08:32:38.439803+0000","last_undegraded":"2024-09-17T08:32:38.439803+0000","last_fullsized":"2024-09-17T08:32:38.439803+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:32:44.984661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.440778+0000","last_change":"2024-09-17T08:32:10.539856+0000","last_active":"2024-09-17T08:32:38.440778+0000","last_peered":"2024-09-17T08:32:38.440778+0000","last_clean":"2024-09-17T08:32:38.440778+0000","last_became_active":"2024-09-17T08:32:10.539566+0000","last_became_peered":"2024-09-17T08:32:10.539566+0000","last_unstale":"2024-09-17T08:32:38.440778+0000","last_undegraded":"2024-09-17T08:32:38.440778+0000","last_fullsized":"2024-09-17T08:32:38.440778+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:49:57.827911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"67'1","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439298+0000","last_change":"2024-09-17T08:32:12.546667+0000","last_active":"2024-09-17T08:32:38.439298+0000","last_peered":"2024-09-17T08:32:38.439298+0000","last_clean":"2024-09-17T08:32:38.439298+0000","last_became_active":"2024-09-17T08:32:12.542401+0000","last_became_peered":"2024-09-17T08:32:12.542401+0000","last_unstale":"2024-09-17T08:32:38.439298+0000","last_undegraded":"2024-09-17T08:32:38.439298+0000","last_fullsized":"2024-09-17T08:32:38.439298+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:40:18.194600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"67'11","reported_seq":38,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441352+0000","last_change":"2024-09-17T08:32:08.529822+0000","last_active":"2024-09-17T08:32:38.441352+0000","last_peered":"2024-09-17T08:32:38.441352+0000","last_clean":"2024-09-17T08:32:38.441352+0000","last_became_active":"2024-09-17T08:32:08.529291+0000","last_became_peered":"2024-09-17T08:32:08.529291+0000","last_unstale":"2024-09-17T08:32:38.441352+0000","last_undegraded":"2024-09-17T08:32:38.441352+0000","last_fullsized":"2024-09-17T08:32:38.441352+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:06:43.050801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441833+0000","last_change":"2024-09-17T08:32:06.516834+0000","last_active":"2024-09-17T08:32:38.441833+0000","last_peered":"2024-09-17T08:32:38.441833+0000","last_clean":"2024-09-17T08:32:38.441833+0000","last_became_active":"2024-09-17T08:32:06.515253+0000","last_became_peered":"2024-09-17T08:32:06.515253+0000","last_unstale":"2024-09-17T08:32:38.441833+0000","last_undegraded":"2024-09-17T08:32:38.441833+0000","last_fullsized":"2024-09-17T08:32:38.441833+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:25.703092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439212+0000","last_change":"2024-09-17T08:32:10.542663+0000","last_active":"2024-09-17T08:32:38.439212+0000","last_peered":"2024-09-17T08:32:38.439212+0000","last_clean":"2024-09-17T08:32:38.439212+0000","last_became_active":"2024-09-17T08:32:10.542393+0000","last_became_peered":"2024-09-17T08:32:10.542393+0000","last_unstale":"2024-09-17T08:32:38.439212+0000","last_undegraded":"2024-09-17T08:32:38.439212+0000","last_fullsized":"2024-09-17T08:32:38.439212+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:12:17.412003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.440930+0000","last_change":"2024-09-17T08:32:12.694792+0000","last_active":"2024-09-17T08:32:38.440930+0000","last_peered":"2024-09-17T08:32:38.440930+0000","last_clean":"2024-09-17T08:32:38.440930+0000","last_became_active":"2024-09-17T08:32:12.694338+0000","last_became_peered":"2024-09-17T08:32:12.694338+0000","last_unstale":"2024-09-17T08:32:38.440930+0000","last_undegraded":"2024-09-17T08:32:38.440930+0000","last_fullsized":"2024-09-17T08:32:38.440930+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:38:31.715989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"67'4","reported_seq":25,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439437+0000","last_change":"2024-09-17T08:32:08.532041+0000","last_active":"2024-09-17T08:32:38.439437+0000","last_peered":"2024-09-17T08:32:38.439437+0000","last_clean":"2024-09-17T08:32:38.439437+0000","last_became_active":"2024-09-17T08:32:08.531798+0000","last_became_peered":"2024-09-17T08:32:08.531798+0000","last_unstale":"2024-09-17T08:32:38.439437+0000","last_undegraded":"2024-09-17T08:32:38.439437+0000","last_fullsized":"2024-09-17T08:32:38.439437+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:43:51.982442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.440829+0000","last_change":"2024-09-17T08:32:06.519248+0000","last_active":"2024-09-17T08:32:38.440829+0000","last_peered":"2024-09-17T08:32:38.440829+0000","last_clean":"2024-09-17T08:32:38.440829+0000","last_became_active":"2024-09-17T08:32:06.518972+0000","last_became_peered":"2024-09-17T08:32:06.518972+0000","last_unstale":"2024-09-17T08:32:38.440829+0000","last_undegraded":"2024-09-17T08:32:38.440829+0000","last_fullsized":"2024-09-17T08:32:38.440829+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:44:47.314633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.445792+0000","last_change":"2024-09-17T08:32:10.538936+0000","last_active":"2024-09-17T08:32:38.445792+0000","last_peered":"2024-09-17T08:32:38.445792+0000","last_clean":"2024-09-17T08:32:38.445792+0000","last_became_active":"2024-09-17T08:32:10.538579+0000","last_became_peered":"2024-09-17T08:32:10.538579+0000","last_unstale":"2024-09-17T08:32:38.445792+0000","last_undegraded":"2024-09-17T08:32:38.445792+0000","last_fullsized":"2024-09-17T08:32:38.445792+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:20:44.477204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439337+0000","last_change":"2024-09-17T08:32:12.549250+0000","last_active":"2024-09-17T08:32:38.439337+0000","last_peered":"2024-09-17T08:32:38.439337+0000","last_clean":"2024-09-17T08:32:38.439337+0000","last_became_active":"2024-09-17T08:32:12.548998+0000","last_became_peered":"2024-09-17T08:32:12.548998+0000","last_unstale":"2024-09-17T08:32:38.439337+0000","last_undegraded":"2024-09-17T08:32:38.439337+0000","last_fullsized":"2024-09-17T08:32:38.439337+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:43:27.021397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.444180+0000","last_change":"2024-09-17T08:32:12.544439+0000","last_active":"2024-09-17T08:32:38.444180+0000","last_peered":"2024-09-17T08:32:38.444180+0000","last_clean":"2024-09-17T08:32:38.444180+0000","last_became_active":"2024-09-17T08:32:12.544146+0000","last_became_peered":"2024-09-17T08:32:12.544146+0000","last_unstale":"2024-09-17T08:32:38.444180+0000","last_undegraded":"2024-09-17T08:32:38.444180+0000","last_fullsized":"2024-09-17T08:32:38.444180+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:03:12.011989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439658+0000","last_change":"2024-09-17T08:32:06.519681+0000","last_active":"2024-09-17T08:32:38.439658+0000","last_peered":"2024-09-17T08:32:38.439658+0000","last_clean":"2024-09-17T08:32:38.439658+0000","last_became_active":"2024-09-17T08:32:06.519097+0000","last_became_peered":"2024-09-17T08:32:06.519097+0000","last_unstale":"2024-09-17T08:32:38.439658+0000","last_undegraded":"2024-09-17T08:32:38.439658+0000","last_fullsized":"2024-09-17T08:32:38.439658+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:12:14.239889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"67'11","reported_seq":38,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.446389+0000","last_change":"2024-09-17T08:32:08.529302+0000","last_active":"2024-09-17T08:32:38.446389+0000","last_peered":"2024-09-17T08:32:38.446389+0000","last_clean":"2024-09-17T08:32:38.446389+0000","last_became_active":"2024-09-17T08:32:08.528998+0000","last_became_peered":"2024-09-17T08:32:08.528998+0000","last_unstale":"2024-09-17T08:32:38.446389+0000","last_undegraded":"2024-09-17T08:32:38.446389+0000","last_fullsized":"2024-09-17T08:32:38.446389+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:51:00.009253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.440937+0000","last_change":"2024-09-17T08:32:10.543482+0000","last_active":"2024-09-17T08:32:38.440937+0000","last_peered":"2024-09-17T08:32:38.440937+0000","last_clean":"2024-09-17T08:32:38.440937+0000","last_became_active":"2024-09-17T08:32:10.543158+0000","last_became_peered":"2024-09-17T08:32:10.543158+0000","last_unstale":"2024-09-17T08:32:38.440937+0000","last_undegraded":"2024-09-17T08:32:38.440937+0000","last_fullsized":"2024-09-17T08:32:38.440937+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:20:53.093873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":392,"ondisk_log_size":392,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":72,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234006,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28376,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716776,"statfs":{"total":95995035648,"available":95965978624,"internally_reserved":0,"allocated":1392640,"data_stored":923806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.437}]},{"osd":1,"last update":"Tue Sep 17 08:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.497}]},{"osd":2,"last update":"Tue Sep 17 08:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.46700000000000003}]},{"osd":3,"last update":"Tue Sep 17 08:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.59199999999999997}]},{"osd":4,"last update":"Tue Sep 17 08:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.60399999999999998}]},{"osd":5,"last update":"Tue Sep 17 08:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.623},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.64900000000000002}]},{"osd":6,"last update":"Tue Sep 17 08:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.63600000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495643,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28344,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716808,"statfs":{"total":95995035648,"available":95966011392,"internally_reserved":0,"allocated":1359872,"data_stored":921424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:32:31 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.60499999999999998}]},{"osd":1,"last update":"Tue Sep 17 08:32:31 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.5},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.63}]},{"osd":2,"last update":"Tue Sep 17 08:32:31 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.47299999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:32:31 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.46400000000000002}]},{"osd":4,"last update":"Tue Sep 17 08:32:31 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.33400000000000002}]},{"osd":5,"last update":"Tue Sep 17 08:32:31 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.37},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.38}]},{"osd":7,"last update":"Tue Sep 17 08:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.40600000000000003}]}]},{"osd":5,"up_from":38,"seq":163208757280,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27800,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717352,"statfs":{"total":95995035648,"available":95966568448,"internally_reserved":0,"allocated":798720,"data_stored":331988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:33:12 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.55400000000000005,"15min":0.56000000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.71599999999999997,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.53800000000000003,"15min":0.54600000000000004},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.75700000000000001,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.44}]},{"osd":1,"last update":"Tue Sep 17 08:33:12 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.55100000000000005,"15min":0.55800000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77500000000000002,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.56799999999999995,"15min":0.57699999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.69999999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.53400000000000003}]},{"osd":2,"last update":"Tue Sep 17 08:33:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57799999999999996,"15min":0.58299999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.74399999999999999,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.56000000000000005,"15min":0.56699999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.72999999999999998,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.56399999999999995}]},{"osd":3,"last update":"Tue Sep 17 08:33:12 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.54500000000000004,"15min":0.55200000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68700000000000006,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56999999999999995,"15min":0.57399999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.72099999999999997,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.42499999999999999}]},{"osd":4,"last update":"Tue Sep 17 08:33:12 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.41899999999999998,"15min":0.41699999999999998},"min":{"1min":0.28299999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.64900000000000002,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.372},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.30599999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.628,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.40699999999999997}]},{"osd":6,"last update":"Tue Sep 17 08:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.49099999999999999}]},{"osd":7,"last update":"Tue Sep 17 08:32:54 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.504}]}]},{"osd":4,"up_from":30,"seq":128849018917,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31892,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713260,"statfs":{"total":95995035648,"available":95962378240,"internally_reserved":0,"allocated":794624,"data_stored":332026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.54100000000000004,"15min":0.54900000000000004},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.51900000000000002,"15min":0.52100000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70599999999999996,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.624}]},{"osd":1,"last update":"Tue Sep 17 08:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52700000000000002,"15min":0.52400000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54700000000000004,"15min":0.54300000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.52700000000000002}]},{"osd":2,"last update":"Tue Sep 17 08:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.54800000000000004,"15min":0.55000000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71199999999999997,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.52000000000000002,"15min":0.52500000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.64500000000000002,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.65500000000000003}]},{"osd":3,"last update":"Tue Sep 17 08:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53200000000000003,"15min":0.52900000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.54300000000000004},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.73499999999999999}]},{"osd":5,"last update":"Tue Sep 17 08:33:13 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.38900000000000001,"15min":0.379},"min":{"1min":0.26500000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.315},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.40899999999999997,"15min":0.39700000000000002},"min":{"1min":0.32100000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.42099999999999999}]},{"osd":6,"last update":"Tue Sep 17 08:32:31 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.442},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.38200000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.46300000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182440,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31908,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713244,"statfs":{"total":95995035648,"available":95962361856,"internally_reserved":0,"allocated":811008,"data_stored":333232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:33:26 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.377,"15min":0.38900000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.375,"5min":0.39100000000000001,"15min":0.40000000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.42599999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:33:26 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.40000000000000002,"15min":0.40600000000000003},"min":{"1min":0.254,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.39800000000000002,"15min":0.40899999999999997},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.28199999999999997}]},{"osd":2,"last update":"Tue Sep 17 08:33:26 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.42999999999999999,"15min":0.44500000000000001},"min":{"1min":0.29999999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.53900000000000003,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.37},{"interface":"front","average":{"1min":0.379,"5min":0.39500000000000002,"15min":0.40400000000000003},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.35799999999999998}]},{"osd":4,"last update":"Tue Sep 17 08:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.40699999999999997}]},{"osd":5,"last update":"Tue Sep 17 08:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.40500000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.47799999999999998}]},{"osd":6,"last update":"Tue Sep 17 08:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.435}]},{"osd":7,"last update":"Tue Sep 17 08:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.48899999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411374,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27780,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717372,"statfs":{"total":95995035648,"available":95966588928,"internally_reserved":0,"allocated":778240,"data_stored":332157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:32:58 2024","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.33100000000000002,"15min":0.32000000000000001},"min":{"1min":0.20899999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.41499999999999998,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.441},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.35299999999999998,"15min":0.34200000000000003},"min":{"1min":0.223,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.49199999999999999,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.629}]},{"osd":1,"last update":"Tue Sep 17 08:32:58 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.34599999999999997,"15min":0.33600000000000002},"min":{"1min":0.23799999999999999,"5min":0.216,"15min":0.216},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.36499999999999999,"15min":0.35699999999999998},"min":{"1min":0.24399999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.45300000000000001,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.5}]},{"osd":3,"last update":"Tue Sep 17 08:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.39900000000000002,"15min":0.40699999999999997},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.41099999999999998,"15min":0.41899999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.54200000000000004,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.54200000000000004}]},{"osd":4,"last update":"Tue Sep 17 08:32:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56200000000000006,"15min":0.55600000000000005},"min":{"1min":0.42299999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54600000000000004,"15min":0.54400000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.71699999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.60899999999999999}]},{"osd":5,"last update":"Tue Sep 17 08:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57199999999999995,"15min":0.56999999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.53200000000000003}]},{"osd":6,"last update":"Tue Sep 17 08:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.69599999999999995}]},{"osd":7,"last update":"Tue Sep 17 08:32:55 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.66600000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":815104,"data_stored":333662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.33500000000000002,"15min":0.32600000000000001},"min":{"1min":0.20200000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.502,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.35099999999999998,"15min":0.33300000000000002},"min":{"1min":0.24099999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.53500000000000003}]},{"osd":2,"last update":"Tue Sep 17 08:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.38100000000000001,"15min":0.375},"min":{"1min":0.20499999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.50600000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.42399999999999999,"15min":0.41999999999999998},"min":{"1min":0.28499999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.437}]},{"osd":3,"last update":"Tue Sep 17 08:33:25 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.39800000000000002,"15min":0.40600000000000003},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.48399999999999999,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.42999999999999999,"15min":0.435},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.53900000000000003,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.48999999999999999}]},{"osd":4,"last update":"Tue Sep 17 08:32:45 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54200000000000004,"15min":0.54000000000000004},"min":{"1min":0.35299999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53600000000000003,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.68200000000000005,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.52000000000000002}]},{"osd":5,"last update":"Tue Sep 17 08:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.60299999999999998,"15min":0.61399999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.502,"5min":0.59099999999999997,"15min":0.60599999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.65800000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.65000000000000002}]},{"osd":6,"last update":"Tue Sep 17 08:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.57099999999999995}]},{"osd":7,"last update":"Tue Sep 17 08:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.57899999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738424,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28380,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716772,"statfs":{"total":95995035648,"available":95965974528,"internally_reserved":0,"allocated":1392640,"data_stored":922642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 17 08:32:35 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.33300000000000002,"15min":0.313},"min":{"1min":0.24399999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.52400000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.33900000000000002,"15min":0.314},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.53600000000000003}]},{"osd":2,"last update":"Tue Sep 17 08:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.40000000000000002,"15min":0.39400000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.50800000000000001,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.40000000000000002,"15min":0.39800000000000002},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.53200000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.89800000000000002}]},{"osd":3,"last update":"Tue Sep 17 08:33:26 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.44400000000000001,"15min":0.45600000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.61799999999999999,"5min":0.629,"15min":0.629},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.46300000000000002,"15min":0.48699999999999999},"min":{"1min":0.28299999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.621,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.64800000000000002}]},{"osd":4,"last update":"Tue Sep 17 08:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.39400000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.73999999999999999,"5min":0.754,"15min":0.754},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57299999999999995,"15min":0.57199999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.83499999999999996}]},{"osd":5,"last update":"Tue Sep 17 08:33:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57799999999999996,"15min":0.57699999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.41099999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.73299999999999998,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.998}]},{"osd":6,"last update":"Tue Sep 17 08:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":1.02},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.67200000000000004}]},{"osd":7,"last update":"Tue Sep 17 08:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.96599999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T08:33:34.904 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-17T08:33:35.100 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:35.145 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:34 smithi170 systemd[1]: Starting Ceph grafana.a for 7862d192-74ce-11ef-bceb-c7b262605968... 2024-09-17T08:33:35.448 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-mon[33271]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:35.448 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-mon[33271]: from='client.24596 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:33:35.449 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 podman[65900]: 2024-09-17 08:33:35.152995405 +0000 UTC m=+0.106747935 container create 1294232fb6913e5048cc7b0afc57c05785244270c58d6c977626836cf86648d4 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., version=8.5, io.k8s.display-name=Red Hat Universal Base Image 8, architecture=x86_64, maintainer=Paul Cuzner , io.openshift.expose-services=, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, distribution-scope=public, vcs-type=git, description=Ceph Grafana Container, build-date=2022-04-29T11:48:11.055897, summary=Grafana Container configured for Ceph mgr/dashboard integration, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.component=ubi8-container, io.openshift.tags=base rhel8, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vendor=Red Hat, Inc., url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, name=ubi8, io.buildah.version=1.32.0, release=239.1651231664) 2024-09-17T08:33:35.449 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 podman[65900]: 2024-09-17 08:33:35.061553919 +0000 UTC m=+0.015306448 image pull 954c08fa618887fe9a6fe572ae541197c77e625b05f4193df7898f5ac649ada9 quay.io/ceph/ceph-grafana:9.4.7 2024-09-17T08:33:35.449 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 podman[65900]: 2024-09-17 08:33:35.374494052 +0000 UTC m=+0.328246585 container init 1294232fb6913e5048cc7b0afc57c05785244270c58d6c977626836cf86648d4 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a, com.redhat.component=ubi8-container, release=239.1651231664, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.k8s.display-name=Red Hat Universal Base Image 8, architecture=x86_64, vcs-type=git, maintainer=Paul Cuzner , build-date=2022-04-29T11:48:11.055897, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.buildah.version=1.32.0, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.openshift.expose-services=, io.openshift.tags=base rhel8, summary=Grafana Container configured for Ceph mgr/dashboard integration, description=Ceph Grafana Container, vendor=Red Hat, Inc., name=ubi8, distribution-scope=public, version=8.5) 2024-09-17T08:33:35.449 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 podman[65900]: 2024-09-17 08:33:35.378936783 +0000 UTC m=+0.332689313 container start 1294232fb6913e5048cc7b0afc57c05785244270c58d6c977626836cf86648d4 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a, vendor=Red Hat, Inc., url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.k8s.display-name=Red Hat Universal Base Image 8, architecture=x86_64, description=Ceph Grafana Container, io.openshift.tags=base rhel8, version=8.5, summary=Grafana Container configured for Ceph mgr/dashboard integration, name=ubi8, distribution-scope=public, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, build-date=2022-04-29T11:48:11.055897, release=239.1651231664, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.expose-services=, com.redhat.component=ubi8-container, vcs-type=git, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, maintainer=Paul Cuzner , io.buildah.version=1.32.0) 2024-09-17T08:33:35.449 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 bash[65900]: 1294232fb6913e5048cc7b0afc57c05785244270c58d6c977626836cf86648d4 2024-09-17T08:33:35.488 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:35 smithi118 ceph-mon[29491]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:35.488 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:35 smithi118 ceph-mon[29491]: from='client.24596 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:33:35.488 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:35 smithi118 ceph-mon[41418]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:35.488 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:35 smithi118 ceph-mon[41418]: from='client.24596 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:33:35.699 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 systemd[1]: Started Ceph grafana.a for 7862d192-74ce-11ef-bceb-c7b262605968. 2024-09-17T08:33:35.699 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=settings t=2024-09-17T08:33:35.534360718Z level=info msg="Starting Grafana" version=9.4.7 commit=4add91f03d branch=HEAD compiled=2023-03-16T23:56:52Z 2024-09-17T08:33:35.699 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=settings t=2024-09-17T08:33:35.534533387Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-09-17T08:33:35.699 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=settings t=2024-09-17T08:33:35.534551058Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-09-17T08:33:35.700 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=settings t=2024-09-17T08:33:35.534557583Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-09-17T08:33:35.700 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=settings t=2024-09-17T08:33:35.534564045Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-09-17T08:33:35.700 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=settings t=2024-09-17T08:33:35.534570242Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-09-17T08:33:35.700 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=settings t=2024-09-17T08:33:35.534576993Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-09-17T08:33:35.700 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=settings t=2024-09-17T08:33:35.534582676Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-09-17T08:33:35.700 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=settings t=2024-09-17T08:33:35.534588356Z level=info msg="Path Home" path=/usr/share/grafana 2024-09-17T08:33:35.700 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=settings t=2024-09-17T08:33:35.534594012Z level=info msg="Path Data" path=/var/lib/grafana 2024-09-17T08:33:35.700 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=settings t=2024-09-17T08:33:35.534599162Z level=info msg="Path Logs" path=/var/log/grafana 2024-09-17T08:33:35.700 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=settings t=2024-09-17T08:33:35.534604314Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-09-17T08:33:35.700 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=settings t=2024-09-17T08:33:35.534609849Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-09-17T08:33:35.700 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=settings t=2024-09-17T08:33:35.534615422Z level=info msg="App mode production" 2024-09-17T08:33:35.700 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=sqlstore t=2024-09-17T08:33:35.534694009Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-09-17T08:33:35.700 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=sqlstore t=2024-09-17T08:33:35.534711235Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-09-17T08:33:35.701 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.53551353Z level=info msg="Starting DB migrations" 2024-09-17T08:33:35.701 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.535584211Z level=info msg="Executing migration" id="create migration_log table" 2024-09-17T08:33:35.701 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.536096862Z level=info msg="Executing migration" id="create user table" 2024-09-17T08:33:35.701 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.536498214Z level=info msg="Executing migration" id="add unique index user.login" 2024-09-17T08:33:35.701 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.537160474Z level=info msg="Executing migration" id="add unique index user.email" 2024-09-17T08:33:35.702 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.53754569Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-09-17T08:33:35.702 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.537909176Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-09-17T08:33:35.702 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.538262622Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-09-17T08:33:35.702 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.538896365Z level=info msg="Executing migration" id="create user table v2" 2024-09-17T08:33:35.702 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.539244127Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-09-17T08:33:35.702 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.539582054Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-09-17T08:33:35.702 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.539954212Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-09-17T08:33:35.702 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.540255735Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-09-17T08:33:35.702 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.540531244Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-09-17T08:33:35.702 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.540870785Z level=info msg="Executing migration" id="Update user table charset" 2024-09-17T08:33:35.702 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.541139046Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-09-17T08:33:35.702 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.541474616Z level=info msg="Executing migration" id="Add missing user data" 2024-09-17T08:33:35.702 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.54175242Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-09-17T08:33:35.702 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.542157195Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-09-17T08:33:35.703 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.542582535Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-09-17T08:33:35.703 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.542955054Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-09-17T08:33:35.703 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.544033532Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-09-17T08:33:35.703 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.544429771Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-09-17T08:33:35.703 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.544818685Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-09-17T08:33:35.703 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.545197966Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-09-17T08:33:35.703 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.545561898Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-09-17T08:33:35.703 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.54593872Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-09-17T08:33:35.703 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.546193532Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-09-17T08:33:35.703 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.546501559Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-09-17T08:33:35.703 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.546803598Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-09-17T08:33:35.703 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.54714594Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-09-17T08:33:35.703 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.547488656Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-09-17T08:33:35.703 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.548073684Z level=info msg="Executing migration" id="create temp_user v2" 2024-09-17T08:33:35.704 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.548431988Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-09-17T08:33:35.704 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.548750819Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-09-17T08:33:35.704 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.54910778Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-09-17T08:33:35.704 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.549451891Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-09-17T08:33:35.704 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.549815785Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-09-17T08:33:35.704 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.550130016Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-09-17T08:33:35.704 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.550427531Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-09-17T08:33:35.704 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.550685161Z level=info msg="Executing migration" id="create star table" 2024-09-17T08:33:35.704 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.551014356Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-09-17T08:33:35.704 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.551387576Z level=info msg="Executing migration" id="create org table v1" 2024-09-17T08:33:35.704 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.551747371Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-09-17T08:33:35.704 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.552144111Z level=info msg="Executing migration" id="create org_user table v1" 2024-09-17T08:33:35.704 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.552464782Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-09-17T08:33:35.704 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.552806483Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-09-17T08:33:35.705 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.553175096Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-09-17T08:33:35.705 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.553523444Z level=info msg="Executing migration" id="Update org table charset" 2024-09-17T08:33:35.705 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.553735872Z level=info msg="Executing migration" id="Update org_user table charset" 2024-09-17T08:33:35.705 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.554015062Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-09-17T08:33:35.705 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.554285541Z level=info msg="Executing migration" id="create dashboard table" 2024-09-17T08:33:35.705 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.554613323Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-09-17T08:33:35.705 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.554972721Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-09-17T08:33:35.705 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.555347769Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-09-17T08:33:35.705 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.555708503Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-09-17T08:33:35.705 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.556079685Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-09-17T08:33:35.705 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.556436834Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-09-17T08:33:35.705 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.557261449Z level=info msg="Executing migration" id="create dashboard v2" 2024-09-17T08:33:35.706 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.557617469Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-09-17T08:33:35.707 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.55803737Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-09-17T08:33:35.707 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.558451469Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-09-17T08:33:35.707 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.558703492Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-09-17T08:33:35.707 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.559056889Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-09-17T08:33:35.707 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.559318754Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-09-17T08:33:35.707 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.559720134Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-09-17T08:33:35.707 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.560166624Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-09-17T08:33:35.707 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.560616389Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-09-17T08:33:35.707 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.561011911Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-09-17T08:33:35.707 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.561494182Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-09-17T08:33:35.708 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.561844641Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-09-17T08:33:35.708 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.562273651Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-09-17T08:33:35.708 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.562509129Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-09-17T08:33:35.708 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.562726976Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-09-17T08:33:35.708 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.563195307Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-09-17T08:33:35.708 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.563647447Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-09-17T08:33:35.708 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.564108741Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-09-17T08:33:35.708 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.564549461Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-09-17T08:33:35.708 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.564792788Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-09-17T08:33:35.708 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.565169192Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-09-17T08:33:35.708 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.565525233Z level=info msg="Executing migration" id="Update dashboard title length" 2024-09-17T08:33:35.708 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.565748955Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-09-17T08:33:35.708 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.566188086Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-09-17T08:33:35.709 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.566534763Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-09-17T08:33:35.709 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.567470217Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-09-17T08:33:35.709 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.567812748Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-09-17T08:33:35.709 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.568190822Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-09-17T08:33:35.709 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.568556503Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-09-17T08:33:35.709 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.568802447Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-09-17T08:33:35.709 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.569128731Z level=info msg="Executing migration" id="Add check_sum column" 2024-09-17T08:33:35.709 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.569601452Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-09-17T08:33:35.709 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.569958293Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-09-17T08:33:35.709 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.570233215Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-09-17T08:33:35.709 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.570462765Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-09-17T08:33:35.709 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.570816077Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-09-17T08:33:35.709 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.571354502Z level=info msg="Executing migration" id="create data_source table" 2024-09-17T08:33:35.710 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.57173203Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-09-17T08:33:35.710 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.572116366Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-09-17T08:33:35.710 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.572497963Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-09-17T08:33:35.710 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.572810806Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-09-17T08:33:35.710 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.573173209Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-09-17T08:33:35.710 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.574143979Z level=info msg="Executing migration" id="create data_source table v2" 2024-09-17T08:33:35.710 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.574511619Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-09-17T08:33:35.710 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.574848364Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-09-17T08:33:35.710 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.575224414Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-09-17T08:33:35.710 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.575507727Z level=info msg="Executing migration" id="Add column with_credentials" 2024-09-17T08:33:35.710 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.57596343Z level=info msg="Executing migration" id="Add secure json data column" 2024-09-17T08:33:35.710 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.576477771Z level=info msg="Executing migration" id="Update data_source table charset" 2024-09-17T08:33:35.710 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.57670032Z level=info msg="Executing migration" id="Update initial version to 1" 2024-09-17T08:33:35.710 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.576953235Z level=info msg="Executing migration" id="Add read_only data column" 2024-09-17T08:33:35.710 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.577434503Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-09-17T08:33:35.711 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.577677038Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-09-17T08:33:35.711 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.577931241Z level=info msg="Executing migration" id="Add uid column" 2024-09-17T08:33:35.711 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.578431097Z level=info msg="Executing migration" id="Update uid value" 2024-09-17T08:33:35.711 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.578671088Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-09-17T08:33:35.711 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.579067843Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-09-17T08:33:35.711 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.579448431Z level=info msg="Executing migration" id="create api_key table" 2024-09-17T08:33:35.711 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.579797804Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-09-17T08:33:35.711 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.580169097Z level=info msg="Executing migration" id="add index api_key.key" 2024-09-17T08:33:35.711 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.580530916Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-09-17T08:33:35.711 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.580936841Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-09-17T08:33:35.711 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.581293689Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-09-17T08:33:35.711 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.581627443Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-09-17T08:33:35.711 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.581964364Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-09-17T08:33:35.712 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.582966701Z level=info msg="Executing migration" id="create api_key table v2" 2024-09-17T08:33:35.712 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.583329165Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-09-17T08:33:35.712 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.583656058Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-09-17T08:33:35.712 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.584018182Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-09-17T08:33:35.712 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.584401953Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-09-17T08:33:35.712 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.584656883Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-09-17T08:33:35.712 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.584945304Z level=info msg="Executing migration" id="Update api_key table charset" 2024-09-17T08:33:35.712 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.585193622Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-09-17T08:33:35.712 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.585647553Z level=info msg="Executing migration" id="Add service account foreign key" 2024-09-17T08:33:35.712 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.586182823Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-09-17T08:33:35.712 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.586450123Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-09-17T08:33:35.712 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.586917691Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-09-17T08:33:35.712 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.587411002Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-09-17T08:33:35.712 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.587721798Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-09-17T08:33:35.713 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.588054779Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-09-17T08:33:35.713 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.588402669Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-09-17T08:33:35.713 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.588772814Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-09-17T08:33:35.713 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.589186436Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-09-17T08:33:35.713 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.589557418Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-09-17T08:33:35.713 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.589781558Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-09-17T08:33:35.713 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.590059831Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-09-17T08:33:35.713 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.590570502Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-09-17T08:33:35.713 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.591107958Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-09-17T08:33:35.713 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.591391717Z level=info msg="Executing migration" id="create quota table v1" 2024-09-17T08:33:35.713 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.591744763Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-09-17T08:33:35.713 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.592150783Z level=info msg="Executing migration" id="Update quota table charset" 2024-09-17T08:33:35.713 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.5924326Z level=info msg="Executing migration" id="create plugin_setting table" 2024-09-17T08:33:35.714 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.592793185Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-09-17T08:33:35.714 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.5931979Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-09-17T08:33:35.714 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.593759348Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-09-17T08:33:35.714 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.594039053Z level=info msg="Executing migration" id="create session table" 2024-09-17T08:33:35.714 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.594440023Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-09-17T08:33:35.714 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.594692221Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-09-17T08:33:35.714 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.594937657Z level=info msg="Executing migration" id="create playlist table v2" 2024-09-17T08:33:35.714 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.595297609Z level=info msg="Executing migration" id="create playlist item table v2" 2024-09-17T08:33:35.714 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.595644161Z level=info msg="Executing migration" id="Update playlist table charset" 2024-09-17T08:33:35.714 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.595882682Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-09-17T08:33:35.714 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.596186719Z level=info msg="Executing migration" id="drop preferences table v2" 2024-09-17T08:33:35.714 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.596431523Z level=info msg="Executing migration" id="drop preferences table v3" 2024-09-17T08:33:35.714 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.596682687Z level=info msg="Executing migration" id="create preferences table v3" 2024-09-17T08:33:35.714 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.597076318Z level=info msg="Executing migration" id="Update preferences table charset" 2024-09-17T08:33:35.715 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.597362682Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-09-17T08:33:35.715 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.597905467Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-09-17T08:33:35.715 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.598189512Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-09-17T08:33:35.715 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.598711858Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-09-17T08:33:35.715 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.599305809Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-09-17T08:33:35.715 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.599580531Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-09-17T08:33:35.715 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.599995719Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-09-17T08:33:35.715 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.600454182Z level=info msg="Executing migration" id="create alert table v1" 2024-09-17T08:33:35.715 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.600873181Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-09-17T08:33:35.715 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.601326094Z level=info msg="Executing migration" id="add index alert state" 2024-09-17T08:33:35.715 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.601709196Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-09-17T08:33:35.715 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.602138573Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-09-17T08:33:35.715 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.602495239Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-09-17T08:33:35.715 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.602898317Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-09-17T08:33:35.715 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.603298164Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-09-17T08:33:35.716 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.604672603Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-09-17T08:33:35.716 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.605048091Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-09-17T08:33:35.716 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.605458179Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-09-17T08:33:35.716 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.605719611Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-09-17T08:33:35.716 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.606096767Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-09-17T08:33:35.716 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.606461895Z level=info msg="Executing migration" id="Add column is_default" 2024-09-17T08:33:35.716 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.60708153Z level=info msg="Executing migration" id="Add column frequency" 2024-09-17T08:33:35.716 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.60767721Z level=info msg="Executing migration" id="Add column send_reminder" 2024-09-17T08:33:35.716 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.608335632Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-17T08:33:35.716 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.608935324Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-17T08:33:35.716 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.609337602Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-17T08:33:35.716 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.609582626Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-17T08:33:35.716 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.60982586Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-17T08:33:35.716 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.610278224Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-17T08:33:35.716 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.610675799Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-17T08:33:35.718 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.611057205Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-17T08:33:35.718 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.611446516Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-17T08:33:35.718 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.611822374Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-17T08:33:35.718 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.612428361Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-17T08:33:35.718 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.613040169Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-17T08:33:35.718 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.613343195Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-17T08:33:35.718 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.613748933Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-17T08:33:35.719 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.614142294Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-17T08:33:35.719 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.614762532Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-17T08:33:35.719 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.615046757Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-17T08:33:35.719 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.615437935Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-17T08:33:35.719 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.615834759Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-17T08:33:35.719 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.61631831Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-17T08:33:35.719 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.616726718Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-17T08:33:35.719 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.617129337Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-17T08:33:35.719 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.61752075Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-17T08:33:35.719 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.617934779Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-17T08:33:35.719 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.618353553Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-17T08:33:35.719 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.618762708Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-17T08:33:35.719 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.619041506Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-17T08:33:35.719 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.619654693Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-17T08:33:35.719 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.620070293Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-17T08:33:35.720 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.6207057Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-17T08:33:35.720 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.621077209Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-17T08:33:35.720 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.621486583Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-17T08:33:35.720 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.621884648Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-17T08:33:35.720 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.623459763Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-17T08:33:35.720 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.623820879Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-17T08:33:35.720 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.624279648Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-17T08:33:35.720 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.624567429Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-17T08:33:35.720 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.624861432Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-17T08:33:35.720 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.625181569Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-17T08:33:35.720 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.625777382Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-17T08:33:35.720 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.626481189Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-17T08:33:35.720 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.626877421Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-17T08:33:35.720 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.627310572Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-17T08:33:35.720 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.627562713Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-17T08:33:35.721 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.628212258Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-17T08:33:35.721 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.62861887Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-17T08:33:35.721 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.628877036Z level=info msg="Executing migration" id="Move region to single row" 2024-09-17T08:33:35.721 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.629243319Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-17T08:33:35.721 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.629593709Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-17T08:33:35.721 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.629966829Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-17T08:33:35.721 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.63036952Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-17T08:33:35.721 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.630726072Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-17T08:33:35.721 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.631111766Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-17T08:33:35.721 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.63149301Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-17T08:33:35.721 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.631744021Z level=info msg="Executing migration" id="create test_data table" 2024-09-17T08:33:35.721 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.632136419Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-17T08:33:35.721 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.632490444Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-17T08:33:35.721 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.632879127Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-17T08:33:35.722 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.633334723Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-17T08:33:35.722 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.63358298Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-17T08:33:35.722 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.633839901Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-17T08:33:35.722 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.634114429Z level=info msg="Executing migration" id="create team table" 2024-09-17T08:33:35.722 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.634469812Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-17T08:33:35.722 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.63489779Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-17T08:33:35.722 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.635302399Z level=info msg="Executing migration" id="create team member table" 2024-09-17T08:33:35.722 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.635656367Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-17T08:33:35.722 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.636057114Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-17T08:33:35.722 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.636512908Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-17T08:33:35.722 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.636958978Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-17T08:33:35.722 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.637635939Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-17T08:33:35.722 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.638305571Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-17T08:33:35.722 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.638980081Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-17T08:33:35.723 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.639357005Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-17T08:33:35.723 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.639773182Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-17T08:33:35.723 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.640188244Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-17T08:33:35.723 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.640581876Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-17T08:33:35.723 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.641031856Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-17T08:33:35.723 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.641434274Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-17T08:33:35.723 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.641818577Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-17T08:33:35.723 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.642209434Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-17T08:33:35.723 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.64261684Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-17T08:33:35.723 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.642875224Z level=info msg="Executing migration" id="create tag table" 2024-09-17T08:33:35.723 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.643237251Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-17T08:33:35.723 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.643625377Z level=info msg="Executing migration" id="create login attempt table" 2024-09-17T08:33:35.723 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.643990249Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-17T08:33:35.723 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.644384717Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-17T08:33:35.723 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.644729029Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-17T08:33:35.723 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.646595566Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-17T08:33:35.724 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.646964703Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-17T08:33:35.724 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.647361281Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-17T08:33:35.724 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.647636213Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-17T08:33:35.724 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.647950418Z level=info msg="Executing migration" id="create user auth table" 2024-09-17T08:33:35.724 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.648307443Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-17T08:33:35.724 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.648695091Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-17T08:33:35.724 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.648938424Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-17T08:33:35.724 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.649664318Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-17T08:33:35.724 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.650423423Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-17T08:33:35.724 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.651161466Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-17T08:33:35.724 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.651930402Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-17T08:33:35.724 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.652327171Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-17T08:33:35.724 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.653036254Z level=info msg="Executing migration" id="create server_lock table" 2024-09-17T08:33:35.725 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.653395804Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-17T08:33:35.725 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.653792108Z level=info msg="Executing migration" id="create user auth token table" 2024-09-17T08:33:35.725 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.654197578Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-17T08:33:35.725 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.654586563Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-17T08:33:35.725 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.654977545Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-17T08:33:35.725 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.655366066Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-17T08:33:35.725 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.656137633Z level=info msg="Executing migration" id="create cache_data table" 2024-09-17T08:33:35.725 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.656511591Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-17T08:33:35.725 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.656908512Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-17T08:33:35.725 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.657295682Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-17T08:33:35.725 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.657686599Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-17T08:33:35.725 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.657931688Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-17T08:33:35.725 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.658203962Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-17T08:33:35.725 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.658599831Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-17T08:33:35.725 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.658993415Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-17T08:33:35.726 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.659385317Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-17T08:33:35.726 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.659622997Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-17T08:33:35.726 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.660001398Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-17T08:33:35.726 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.660379629Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-17T08:33:35.726 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.660743366Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-17T08:33:35.726 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.661167595Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-17T08:33:35.726 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.661934118Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-17T08:33:35.726 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.662342166Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-17T08:33:35.726 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.662598754Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-17T08:33:35.726 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.662946526Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-17T08:33:35.726 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.663350937Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-17T08:33:35.726 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.66372038Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-17T08:33:35.726 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.66399625Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-17T08:33:35.726 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.664445854Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-17T08:33:35.726 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.664802254Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-17T08:33:35.727 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.665231234Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-17T08:33:35.727 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.665623453Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-17T08:33:35.727 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.666403468Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-17T08:33:35.727 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.666796049Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-17T08:33:35.727 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.667188048Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-17T08:33:35.727 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.670568631Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-17T08:33:35.727 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.673871579Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-17T08:33:35.727 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.674289631Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-17T08:33:35.727 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.674652874Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-17T08:33:35.727 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.675442932Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-17T08:33:35.727 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.675831878Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-17T08:33:35.727 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.676238297Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-17T08:33:35.727 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.676656127Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-17T08:33:35.727 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.677082039Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-17T08:33:35.727 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.677350663Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-17T08:33:35.728 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.678128421Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-17T08:33:35.728 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.678897504Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-17T08:33:35.728 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.679664742Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-17T08:33:35.728 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.680059349Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-17T08:33:35.728 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.680454422Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-17T08:33:35.728 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.681256386Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-17T08:33:35.728 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.682060777Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-17T08:33:35.728 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.682461331Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-17T08:33:35.728 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.683280603Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-17T08:33:35.728 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.684068463Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-17T08:33:35.728 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.684359905Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-17T08:33:35.728 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.684771367Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-17T08:33:35.728 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.685195215Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-17T08:33:35.728 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.685594888Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-17T08:33:35.728 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.685831675Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-17T08:33:35.729 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.686651694Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-17T08:33:35.729 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.687529906Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-17T08:33:35.729 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.688385961Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-17T08:33:35.729 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.689195847Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-17T08:33:35.729 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.689994701Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-17T08:33:35.729 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.690270856Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-17T08:33:35.729 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.690629362Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-17T08:33:35.729 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.69144927Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-17T08:33:35.729 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.691722774Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-17T08:33:35.729 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.692602671Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-17T08:33:35.729 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.693010191Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-17T08:33:35.729 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.693822205Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-17T08:33:35.729 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.694194448Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-17T08:33:35.729 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.694588274Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-17T08:33:35.729 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.695432965Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-17T08:33:35.730 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.695801036Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-17T08:33:35.730 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.696212683Z level=info msg="Executing migration" id="create alert_image table" 2024-09-17T08:33:35.730 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.696573019Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-17T08:33:35.730 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.696989873Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-17T08:33:35.730 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.697265517Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-17T08:33:35.730 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.697633059Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-17T08:33:35.730 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.698016673Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-17T08:33:35.730 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.698093471Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-17T08:33:35.730 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.698329165Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-17T08:33:35.730 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.698607388Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-17T08:33:35.730 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.698960172Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-09-17T08:33:35.730 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.699072364Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-09-17T08:33:35.731 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.699174632Z level=info msg="alerts found to migrate" alerts=0 2024-09-17T08:33:35.731 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.69950973Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-17T08:33:35.731 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.699934659Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-17T08:33:35.731 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.700354277Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-17T08:33:35.731 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.700715306Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-17T08:33:35.731 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.701143592Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-17T08:33:35.731 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.701608289Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-17T08:33:35.731 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.701841448Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-17T08:33:35.731 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.702202075Z level=info msg="Executing migration" id="create data_keys table" 2024-09-17T08:33:35.731 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.702577245Z level=info msg="Executing migration" id="create secrets table" 2024-09-17T08:33:35.731 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.7029726Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-17T08:33:35.731 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.707203483Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-17T08:33:35.731 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.70811805Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-17T08:33:35.731 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.708399811Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-17T08:33:35.731 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.712743007Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-17T08:33:36.145 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.717106236Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-17T08:33:36.146 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.717484572Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-17T08:33:36.146 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.718083849Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-17T08:33:36.146 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.718408817Z level=info msg="Executing migration" id="create permission table" 2024-09-17T08:33:36.146 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.718771758Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-17T08:33:36.146 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.719178646Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-17T08:33:36.146 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.719610518Z level=info msg="Executing migration" id="create role table" 2024-09-17T08:33:36.146 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.719995586Z level=info msg="Executing migration" id="add column display_name" 2024-09-17T08:33:36.146 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.720896026Z level=info msg="Executing migration" id="add column group_name" 2024-09-17T08:33:36.146 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.721789251Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-17T08:33:36.146 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.722218222Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-17T08:33:36.146 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.722697829Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-17T08:33:36.146 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.723128272Z level=info msg="Executing migration" id="create team role table" 2024-09-17T08:33:36.146 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.723524113Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-17T08:33:36.147 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.723973868Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-17T08:33:36.147 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.724459952Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-17T08:33:36.147 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.724872921Z level=info msg="Executing migration" id="create user role table" 2024-09-17T08:33:36.147 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.725249999Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-17T08:33:36.147 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.725665041Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-17T08:33:36.147 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.726092015Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-17T08:33:36.147 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.726501144Z level=info msg="Executing migration" id="create builtin role table" 2024-09-17T08:33:36.147 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.726899145Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-17T08:33:36.147 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.727315751Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-17T08:33:36.147 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.727741503Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-17T08:33:36.147 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.728683486Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-17T08:33:36.147 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.729147671Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-17T08:33:36.147 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.729568857Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-17T08:33:36.147 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.729961674Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-17T08:33:36.148 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.730372377Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-17T08:33:36.148 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.730730326Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-17T08:33:36.148 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.731158127Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-17T08:33:36.148 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.732094266Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-17T08:33:36.148 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.732470116Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-17T08:33:36.148 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.732877602Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-17T08:33:36.148 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.733166265Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-17T08:33:36.148 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.733523255Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-17T08:33:36.148 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.733940026Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-17T08:33:36.149 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.734287515Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-17T08:33:36.149 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.734542529Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-17T08:33:36.149 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.734829784Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-17T08:33:36.149 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.735189415Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-17T08:33:36.149 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.735605256Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-17T08:33:36.150 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.736535992Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-17T08:33:36.150 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.736811279Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-17T08:33:36.150 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.737209306Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-17T08:33:36.150 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.737623158Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-17T08:33:36.150 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.738055857Z level=info msg="Executing migration" id="add correlation config column" 2024-09-17T08:33:36.150 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.739000282Z level=info msg="Executing migration" id="create entity_events table" 2024-09-17T08:33:36.150 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.739399593Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-17T08:33:36.150 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.739775023Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-17T08:33:36.150 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.739864357Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-17T08:33:36.150 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.740136175Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-17T08:33:36.150 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.740224609Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-17T08:33:36.150 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.740465911Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-17T08:33:36.150 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.740825236Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-17T08:33:36.150 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.741211936Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-17T08:33:36.150 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.741622578Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-17T08:33:36.151 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.742051937Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-17T08:33:36.151 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.742445756Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-17T08:33:36.151 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.742850674Z level=info msg="Executing migration" id="Drop public config table" 2024-09-17T08:33:36.151 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.743325268Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-17T08:33:36.151 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.743693311Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-17T08:33:36.151 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.744120312Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-17T08:33:36.151 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.744557492Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-17T08:33:36.151 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.744976581Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-17T08:33:36.151 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.747933412Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-17T08:33:36.151 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.748913153Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-17T08:33:36.151 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.749871028Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-17T08:33:36.151 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.750173972Z level=info msg="Executing migration" id="add share column" 2024-09-17T08:33:36.151 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.751164788Z level=info msg="Executing migration" id="create default alerting folders" 2024-09-17T08:33:36.151 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.751594113Z level=info msg="Executing migration" id="create file table" 2024-09-17T08:33:36.151 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.752028782Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-17T08:33:36.151 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.752476011Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-17T08:33:36.152 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.752950332Z level=info msg="Executing migration" id="create file_meta table" 2024-09-17T08:33:36.152 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.753335193Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-17T08:33:36.152 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.753774889Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-17T08:33:36.152 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.754045464Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-17T08:33:36.152 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.754416622Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-17T08:33:36.152 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.754683889Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-17T08:33:36.152 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.755136812Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-17T08:33:36.152 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.756149049Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-17T08:33:36.152 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.75643427Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-17T08:33:36.152 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.756849886Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-17T08:33:36.152 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.757221259Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-17T08:33:36.152 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.757482777Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-17T08:33:36.152 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.75779984Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-17T08:33:36.152 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.758777552Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-17T08:33:36.153 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.759749789Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-17T08:33:36.153 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.760160077Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-17T08:33:36.153 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.770020447Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-17T08:33:36.153 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.770447343Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-17T08:33:36.153 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.770868564Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-17T08:33:36.153 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.774311936Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-17T08:33:36.153 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.7746093Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-17T08:33:36.153 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.774949257Z level=info msg="Executing migration" id="create folder table" 2024-09-17T08:33:36.153 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.775325744Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-17T08:33:36.153 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.775736122Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-17T08:33:36.153 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.776170221Z level=info msg="Executing migration" id="Update folder title length" 2024-09-17T08:33:36.153 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.776429647Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-17T08:33:36.153 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=migrator t=2024-09-17T08:33:35.776832927Z level=info msg="migrations completed" performed=485 skipped=0 duration=241.265977ms 2024-09-17T08:33:36.153 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=sqlstore t=2024-09-17T08:33:35.777256651Z level=info msg="Created default organization" 2024-09-17T08:33:36.153 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=plugin.loader t=2024-09-17T08:33:35.833992288Z level=info msg="Plugin registered" pluginID=input 2024-09-17T08:33:36.154 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=plugin.loader t=2024-09-17T08:33:35.86246727Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-09-17T08:33:36.154 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=plugin.loader t=2024-09-17T08:33:35.862487362Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-09-17T08:33:36.154 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=secrets t=2024-09-17T08:33:35.862523444Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-17T08:33:36.154 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=query_data t=2024-09-17T08:33:35.864188836Z level=info msg="Query Service initialization" 2024-09-17T08:33:36.154 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:35 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=live.push_http t=2024-09-17T08:33:35.866407205Z level=info msg="Live Push Gateway initialization" 2024-09-17T08:33:36.509 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:36 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:36.509 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:36 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:36.510 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:36 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:36.510 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:36 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:36.510 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:36 smithi118 ceph-mon[29491]: Deploying daemon node-exporter.a on smithi118 2024-09-17T08:33:36.510 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:36 smithi118 ceph-mon[29491]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:36.764 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:36 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:36.765 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:36 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:36.765 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:36 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:36.765 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:36 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:36.765 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:36 smithi170 ceph-mon[33271]: Deploying daemon node-exporter.a on smithi118 2024-09-17T08:33:36.765 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:36 smithi170 ceph-mon[33271]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:36.765 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:36 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=infra.usagestats.collector t=2024-09-17T08:33:36.59801056Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-17T08:33:36.765 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:36 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=provisioning.datasources t=2024-09-17T08:33:36.652210056Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-17T08:33:36.765 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:36 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=provisioning.datasources t=2024-09-17T08:33:36.684202001Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-09-17T08:33:36.765 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:36 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=provisioning.alerting t=2024-09-17T08:33:36.705573486Z level=info msg="starting to provision alerting" 2024-09-17T08:33:36.765 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:36 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=provisioning.alerting t=2024-09-17T08:33:36.705597831Z level=info msg="finished to provision alerting" 2024-09-17T08:33:36.766 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:36 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=ngalert.state.manager t=2024-09-17T08:33:36.705715469Z level=info msg="Warming state cache for startup" 2024-09-17T08:33:36.766 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:36 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=grafanaStorageLogger t=2024-09-17T08:33:36.706009479Z level=info msg="storage starting" 2024-09-17T08:33:36.766 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:36 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=http.server t=2024-09-17T08:33:36.707754266Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-17T08:33:36.766 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:36 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=sqlstore.transactions t=2024-09-17T08:33:36.716235145Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-17T08:33:36.766 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:36 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=ngalert.state.manager t=2024-09-17T08:33:36.716312073Z level=info msg="State cache has been initialized" states=0 duration=10.594939ms 2024-09-17T08:33:36.766 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:36 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=ngalert.multiorg.alertmanager t=2024-09-17T08:33:36.716356827Z level=info msg="starting MultiOrg Alertmanager" 2024-09-17T08:33:36.766 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:36 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=ticker t=2024-09-17T08:33:36.716383382Z level=info msg=starting first_tick=2024-09-17T08:33:40Z 2024-09-17T08:33:36.766 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:36 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=sqlstore.transactions t=2024-09-17T08:33:36.765229179Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-17T08:33:36.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:36 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:36.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:36 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:36.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:36 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:36.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:36 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:36.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:36 smithi118 ceph-mon[41418]: Deploying daemon node-exporter.a on smithi118 2024-09-17T08:33:36.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:36 smithi118 ceph-mon[41418]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:37.123 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:33:37.125 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-09-17T08:33:37.145 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:36 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=sqlstore.transactions t=2024-09-17T08:33:36.776638819Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-17T08:33:37.145 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:33:36 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a[65915]: logger=sqlstore.transactions t=2024-09-17T08:33:36.814392898Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-17T08:33:37.449 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:33:37 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:33:37] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:33:37.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:37 smithi118 ceph-mon[29491]: from='client.24602 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:33:37.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:37 smithi118 ceph-mon[41418]: from='client.24602 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:33:37.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:37 smithi170 ceph-mon[33271]: from='client.24602 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:33:38.429 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-09-17T08:33:36.063503+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596246,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":1004,"num_read_kb":850,"num_write":609,"num_write_kb":2459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":1771682,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":598,"ondisk_log_size":598,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":232296,"kb_used_data":7952,"kb_used_omap":12,"kb_used_meta":224243,"kb_avail":749728920,"statfs":{"total":767960285184,"available":767722414080,"internally_reserved":0,"allocated":8142848,"data_stored":4430937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":229625401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002808"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.440752+0000","last_change":"2024-09-17T08:32:06.517049+0000","last_active":"2024-09-17T08:32:38.440752+0000","last_peered":"2024-09-17T08:32:38.440752+0000","last_clean":"2024-09-17T08:32:38.440752+0000","last_became_active":"2024-09-17T08:32:06.516602+0000","last_became_peered":"2024-09-17T08:32:06.516602+0000","last_unstale":"2024-09-17T08:32:38.440752+0000","last_undegraded":"2024-09-17T08:32:38.440752+0000","last_fullsized":"2024-09-17T08:32:38.440752+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:33:39.991581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"67'9","reported_seq":35,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443204+0000","last_change":"2024-09-17T08:32:08.530330+0000","last_active":"2024-09-17T08:32:38.443204+0000","last_peered":"2024-09-17T08:32:38.443204+0000","last_clean":"2024-09-17T08:32:38.443204+0000","last_became_active":"2024-09-17T08:32:08.530083+0000","last_became_peered":"2024-09-17T08:32:08.530083+0000","last_unstale":"2024-09-17T08:32:38.443204+0000","last_undegraded":"2024-09-17T08:32:38.443204+0000","last_fullsized":"2024-09-17T08:32:38.443204+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:29:13.722157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.444550+0000","last_change":"2024-09-17T08:32:10.538280+0000","last_active":"2024-09-17T08:32:38.444550+0000","last_peered":"2024-09-17T08:32:38.444550+0000","last_clean":"2024-09-17T08:32:38.444550+0000","last_became_active":"2024-09-17T08:32:10.537573+0000","last_became_peered":"2024-09-17T08:32:10.537573+0000","last_unstale":"2024-09-17T08:32:38.444550+0000","last_undegraded":"2024-09-17T08:32:38.444550+0000","last_fullsized":"2024-09-17T08:32:38.444550+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:35:24.010164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443333+0000","last_change":"2024-09-17T08:32:12.543966+0000","last_active":"2024-09-17T08:32:38.443333+0000","last_peered":"2024-09-17T08:32:38.443333+0000","last_clean":"2024-09-17T08:32:38.443333+0000","last_became_active":"2024-09-17T08:32:12.543687+0000","last_became_peered":"2024-09-17T08:32:12.543687+0000","last_unstale":"2024-09-17T08:32:38.443333+0000","last_undegraded":"2024-09-17T08:32:38.443333+0000","last_fullsized":"2024-09-17T08:32:38.443333+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:23:01.598906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.440694+0000","last_change":"2024-09-17T08:32:12.694907+0000","last_active":"2024-09-17T08:32:38.440694+0000","last_peered":"2024-09-17T08:32:38.440694+0000","last_clean":"2024-09-17T08:32:38.440694+0000","last_became_active":"2024-09-17T08:32:12.694607+0000","last_became_peered":"2024-09-17T08:32:12.694607+0000","last_unstale":"2024-09-17T08:32:38.440694+0000","last_undegraded":"2024-09-17T08:32:38.440694+0000","last_fullsized":"2024-09-17T08:32:38.440694+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:34.532157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.440476+0000","last_change":"2024-09-17T08:32:06.519591+0000","last_active":"2024-09-17T08:32:38.440476+0000","last_peered":"2024-09-17T08:32:38.440476+0000","last_clean":"2024-09-17T08:32:38.440476+0000","last_became_active":"2024-09-17T08:32:06.519301+0000","last_became_peered":"2024-09-17T08:32:06.519301+0000","last_unstale":"2024-09-17T08:32:38.440476+0000","last_undegraded":"2024-09-17T08:32:38.440476+0000","last_fullsized":"2024-09-17T08:32:38.440476+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:18.417256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"67'15","reported_seq":44,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441441+0000","last_change":"2024-09-17T08:32:08.524076+0000","last_active":"2024-09-17T08:32:38.441441+0000","last_peered":"2024-09-17T08:32:38.441441+0000","last_clean":"2024-09-17T08:32:38.441441+0000","last_became_active":"2024-09-17T08:32:08.523778+0000","last_became_peered":"2024-09-17T08:32:08.523778+0000","last_unstale":"2024-09-17T08:32:38.441441+0000","last_undegraded":"2024-09-17T08:32:38.441441+0000","last_fullsized":"2024-09-17T08:32:38.441441+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:49:14.449660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443174+0000","last_change":"2024-09-17T08:32:10.543440+0000","last_active":"2024-09-17T08:32:38.443174+0000","last_peered":"2024-09-17T08:32:38.443174+0000","last_clean":"2024-09-17T08:32:38.443174+0000","last_became_active":"2024-09-17T08:32:10.542455+0000","last_became_peered":"2024-09-17T08:32:10.542455+0000","last_unstale":"2024-09-17T08:32:38.443174+0000","last_undegraded":"2024-09-17T08:32:38.443174+0000","last_fullsized":"2024-09-17T08:32:38.443174+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:27:52.637574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441912+0000","last_change":"2024-09-17T08:32:06.514230+0000","last_active":"2024-09-17T08:32:38.441912+0000","last_peered":"2024-09-17T08:32:38.441912+0000","last_clean":"2024-09-17T08:32:38.441912+0000","last_became_active":"2024-09-17T08:32:06.513655+0000","last_became_peered":"2024-09-17T08:32:06.513655+0000","last_unstale":"2024-09-17T08:32:38.441912+0000","last_undegraded":"2024-09-17T08:32:38.441912+0000","last_fullsized":"2024-09-17T08:32:38.441912+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:35:35.109613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"67'9","reported_seq":35,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443589+0000","last_change":"2024-09-17T08:32:08.533286+0000","last_active":"2024-09-17T08:32:38.443589+0000","last_peered":"2024-09-17T08:32:38.443589+0000","last_clean":"2024-09-17T08:32:38.443589+0000","last_became_active":"2024-09-17T08:32:08.533023+0000","last_became_peered":"2024-09-17T08:32:08.533023+0000","last_unstale":"2024-09-17T08:32:38.443589+0000","last_undegraded":"2024-09-17T08:32:38.443589+0000","last_fullsized":"2024-09-17T08:32:38.443589+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:56:39.845687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441532+0000","last_change":"2024-09-17T08:32:10.541683+0000","last_active":"2024-09-17T08:32:38.441532+0000","last_peered":"2024-09-17T08:32:38.441532+0000","last_clean":"2024-09-17T08:32:38.441532+0000","last_became_active":"2024-09-17T08:32:10.541279+0000","last_became_peered":"2024-09-17T08:32:10.541279+0000","last_unstale":"2024-09-17T08:32:38.441532+0000","last_undegraded":"2024-09-17T08:32:38.441532+0000","last_fullsized":"2024-09-17T08:32:38.441532+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:11:16.225567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439371+0000","last_change":"2024-09-17T08:32:12.548178+0000","last_active":"2024-09-17T08:32:38.439371+0000","last_peered":"2024-09-17T08:32:38.439371+0000","last_clean":"2024-09-17T08:32:38.439371+0000","last_became_active":"2024-09-17T08:32:12.546998+0000","last_became_peered":"2024-09-17T08:32:12.546998+0000","last_unstale":"2024-09-17T08:32:38.439371+0000","last_undegraded":"2024-09-17T08:32:38.439371+0000","last_fullsized":"2024-09-17T08:32:38.439371+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:40:38.518257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.442167+0000","last_change":"2024-09-17T08:32:06.517632+0000","last_active":"2024-09-17T08:32:38.442167+0000","last_peered":"2024-09-17T08:32:38.442167+0000","last_clean":"2024-09-17T08:32:38.442167+0000","last_became_active":"2024-09-17T08:32:06.517334+0000","last_became_peered":"2024-09-17T08:32:06.517334+0000","last_unstale":"2024-09-17T08:32:38.442167+0000","last_undegraded":"2024-09-17T08:32:38.442167+0000","last_fullsized":"2024-09-17T08:32:38.442167+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:50:35.513812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"67'5","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443428+0000","last_change":"2024-09-17T08:32:08.532942+0000","last_active":"2024-09-17T08:32:38.443428+0000","last_peered":"2024-09-17T08:32:38.443428+0000","last_clean":"2024-09-17T08:32:38.443428+0000","last_became_active":"2024-09-17T08:32:08.532650+0000","last_became_peered":"2024-09-17T08:32:08.532650+0000","last_unstale":"2024-09-17T08:32:38.443428+0000","last_undegraded":"2024-09-17T08:32:38.443428+0000","last_fullsized":"2024-09-17T08:32:38.443428+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:48:47.558738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439344+0000","last_change":"2024-09-17T08:32:10.543302+0000","last_active":"2024-09-17T08:32:38.439344+0000","last_peered":"2024-09-17T08:32:38.439344+0000","last_clean":"2024-09-17T08:32:38.439344+0000","last_became_active":"2024-09-17T08:32:10.543029+0000","last_became_peered":"2024-09-17T08:32:10.543029+0000","last_unstale":"2024-09-17T08:32:38.439344+0000","last_undegraded":"2024-09-17T08:32:38.439344+0000","last_fullsized":"2024-09-17T08:32:38.439344+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:50:51.167211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441449+0000","last_change":"2024-09-17T08:32:12.544708+0000","last_active":"2024-09-17T08:32:38.441449+0000","last_peered":"2024-09-17T08:32:38.441449+0000","last_clean":"2024-09-17T08:32:38.441449+0000","last_became_active":"2024-09-17T08:32:12.544384+0000","last_became_peered":"2024-09-17T08:32:12.544384+0000","last_unstale":"2024-09-17T08:32:38.441449+0000","last_undegraded":"2024-09-17T08:32:38.441449+0000","last_fullsized":"2024-09-17T08:32:38.441449+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:50:27.675858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443915+0000","last_change":"2024-09-17T08:32:12.694656+0000","last_active":"2024-09-17T08:32:38.443915+0000","last_peered":"2024-09-17T08:32:38.443915+0000","last_clean":"2024-09-17T08:32:38.443915+0000","last_became_active":"2024-09-17T08:32:12.694380+0000","last_became_peered":"2024-09-17T08:32:12.694380+0000","last_unstale":"2024-09-17T08:32:38.443915+0000","last_undegraded":"2024-09-17T08:32:38.443915+0000","last_fullsized":"2024-09-17T08:32:38.443915+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:29:59.529947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.440713+0000","last_change":"2024-09-17T08:32:06.516203+0000","last_active":"2024-09-17T08:32:38.440713+0000","last_peered":"2024-09-17T08:32:38.440713+0000","last_clean":"2024-09-17T08:32:38.440713+0000","last_became_active":"2024-09-17T08:32:06.515783+0000","last_became_peered":"2024-09-17T08:32:06.515783+0000","last_unstale":"2024-09-17T08:32:38.440713+0000","last_undegraded":"2024-09-17T08:32:38.440713+0000","last_fullsized":"2024-09-17T08:32:38.440713+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:16:53.826036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"67'15","reported_seq":44,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.444825+0000","last_change":"2024-09-17T08:32:08.528844+0000","last_active":"2024-09-17T08:32:38.444825+0000","last_peered":"2024-09-17T08:32:38.444825+0000","last_clean":"2024-09-17T08:32:38.444825+0000","last_became_active":"2024-09-17T08:32:08.528562+0000","last_became_peered":"2024-09-17T08:32:08.528562+0000","last_unstale":"2024-09-17T08:32:38.444825+0000","last_undegraded":"2024-09-17T08:32:38.444825+0000","last_fullsized":"2024-09-17T08:32:38.444825+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:25:25.572514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.444476+0000","last_change":"2024-09-17T08:32:10.546518+0000","last_active":"2024-09-17T08:32:38.444476+0000","last_peered":"2024-09-17T08:32:38.444476+0000","last_clean":"2024-09-17T08:32:38.444476+0000","last_became_active":"2024-09-17T08:32:10.546204+0000","last_became_peered":"2024-09-17T08:32:10.546204+0000","last_unstale":"2024-09-17T08:32:38.444476+0000","last_undegraded":"2024-09-17T08:32:38.444476+0000","last_fullsized":"2024-09-17T08:32:38.444476+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:32:51.438201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441069+0000","last_change":"2024-09-17T08:32:12.694565+0000","last_active":"2024-09-17T08:32:38.441069+0000","last_peered":"2024-09-17T08:32:38.441069+0000","last_clean":"2024-09-17T08:32:38.441069+0000","last_became_active":"2024-09-17T08:32:12.694235+0000","last_became_peered":"2024-09-17T08:32:12.694235+0000","last_unstale":"2024-09-17T08:32:38.441069+0000","last_undegraded":"2024-09-17T08:32:38.441069+0000","last_fullsized":"2024-09-17T08:32:38.441069+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:30:07.758747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443769+0000","last_change":"2024-09-17T08:32:06.515516+0000","last_active":"2024-09-17T08:32:38.443769+0000","last_peered":"2024-09-17T08:32:38.443769+0000","last_clean":"2024-09-17T08:32:38.443769+0000","last_became_active":"2024-09-17T08:32:06.515246+0000","last_became_peered":"2024-09-17T08:32:06.515246+0000","last_unstale":"2024-09-17T08:32:38.443769+0000","last_undegraded":"2024-09-17T08:32:38.443769+0000","last_fullsized":"2024-09-17T08:32:38.443769+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:40.273522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"67'12","reported_seq":42,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.440045+0000","last_change":"2024-09-17T08:32:08.528795+0000","last_active":"2024-09-17T08:32:38.440045+0000","last_peered":"2024-09-17T08:32:38.440045+0000","last_clean":"2024-09-17T08:32:38.440045+0000","last_became_active":"2024-09-17T08:32:08.528471+0000","last_became_peered":"2024-09-17T08:32:08.528471+0000","last_unstale":"2024-09-17T08:32:38.440045+0000","last_undegraded":"2024-09-17T08:32:38.440045+0000","last_fullsized":"2024-09-17T08:32:38.440045+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:55.620995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443600+0000","last_change":"2024-09-17T08:32:10.536262+0000","last_active":"2024-09-17T08:32:38.443600+0000","last_peered":"2024-09-17T08:32:38.443600+0000","last_clean":"2024-09-17T08:32:38.443600+0000","last_became_active":"2024-09-17T08:32:10.535942+0000","last_became_peered":"2024-09-17T08:32:10.535942+0000","last_unstale":"2024-09-17T08:32:38.443600+0000","last_undegraded":"2024-09-17T08:32:38.443600+0000","last_fullsized":"2024-09-17T08:32:38.443600+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:10:52.244174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"67'1","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439530+0000","last_change":"2024-09-17T08:32:12.550172+0000","last_active":"2024-09-17T08:32:38.439530+0000","last_peered":"2024-09-17T08:32:38.439530+0000","last_clean":"2024-09-17T08:32:38.439530+0000","last_became_active":"2024-09-17T08:32:12.549901+0000","last_became_peered":"2024-09-17T08:32:12.549901+0000","last_unstale":"2024-09-17T08:32:38.439530+0000","last_undegraded":"2024-09-17T08:32:38.439530+0000","last_fullsized":"2024-09-17T08:32:38.439530+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:03:14.243693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.444733+0000","last_change":"2024-09-17T08:32:06.523880+0000","last_active":"2024-09-17T08:32:38.444733+0000","last_peered":"2024-09-17T08:32:38.444733+0000","last_clean":"2024-09-17T08:32:38.444733+0000","last_became_active":"2024-09-17T08:32:06.523220+0000","last_became_peered":"2024-09-17T08:32:06.523220+0000","last_unstale":"2024-09-17T08:32:38.444733+0000","last_undegraded":"2024-09-17T08:32:38.444733+0000","last_fullsized":"2024-09-17T08:32:38.444733+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:51:58.865000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"67'10","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441017+0000","last_change":"2024-09-17T08:32:08.531794+0000","last_active":"2024-09-17T08:32:38.441017+0000","last_peered":"2024-09-17T08:32:38.441017+0000","last_clean":"2024-09-17T08:32:38.441017+0000","last_became_active":"2024-09-17T08:32:08.531427+0000","last_became_peered":"2024-09-17T08:32:08.531427+0000","last_unstale":"2024-09-17T08:32:38.441017+0000","last_undegraded":"2024-09-17T08:32:38.441017+0000","last_fullsized":"2024-09-17T08:32:38.441017+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:56.671609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"67'8","reported_seq":46,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T08:33:29.923745+0000","last_change":"2024-09-17T08:32:10.539394+0000","last_active":"2024-09-17T08:33:29.923745+0000","last_peered":"2024-09-17T08:33:29.923745+0000","last_clean":"2024-09-17T08:33:29.923745+0000","last_became_active":"2024-09-17T08:32:10.539094+0000","last_became_peered":"2024-09-17T08:32:10.539094+0000","last_unstale":"2024-09-17T08:33:29.923745+0000","last_undegraded":"2024-09-17T08:33:29.923745+0000","last_fullsized":"2024-09-17T08:33:29.923745+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:59:30.658834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"67'15","reported_seq":44,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.446042+0000","last_change":"2024-09-17T08:32:08.533283+0000","last_active":"2024-09-17T08:32:38.446042+0000","last_peered":"2024-09-17T08:32:38.446042+0000","last_clean":"2024-09-17T08:32:38.446042+0000","last_became_active":"2024-09-17T08:32:08.532988+0000","last_became_peered":"2024-09-17T08:32:08.532988+0000","last_unstale":"2024-09-17T08:32:38.446042+0000","last_undegraded":"2024-09-17T08:32:38.446042+0000","last_fullsized":"2024-09-17T08:32:38.446042+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:28.483744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"60'1","reported_seq":26,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.440107+0000","last_change":"2024-09-17T08:32:06.519832+0000","last_active":"2024-09-17T08:32:38.440107+0000","last_peered":"2024-09-17T08:32:38.440107+0000","last_clean":"2024-09-17T08:32:38.440107+0000","last_became_active":"2024-09-17T08:32:06.519439+0000","last_became_peered":"2024-09-17T08:32:06.519439+0000","last_unstale":"2024-09-17T08:32:38.440107+0000","last_undegraded":"2024-09-17T08:32:38.440107+0000","last_fullsized":"2024-09-17T08:32:38.440107+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:20:54.535189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"67'8","reported_seq":43,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T08:33:29.923775+0000","last_change":"2024-09-17T08:32:10.543873+0000","last_active":"2024-09-17T08:33:29.923775+0000","last_peered":"2024-09-17T08:33:29.923775+0000","last_clean":"2024-09-17T08:33:29.923775+0000","last_became_active":"2024-09-17T08:32:10.543549+0000","last_became_peered":"2024-09-17T08:32:10.543549+0000","last_unstale":"2024-09-17T08:33:29.923775+0000","last_undegraded":"2024-09-17T08:33:29.923775+0000","last_fullsized":"2024-09-17T08:33:29.923775+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:27:40.672094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441601+0000","last_change":"2024-09-17T08:32:12.545361+0000","last_active":"2024-09-17T08:32:38.441601+0000","last_peered":"2024-09-17T08:32:38.441601+0000","last_clean":"2024-09-17T08:32:38.441601+0000","last_became_active":"2024-09-17T08:32:12.545077+0000","last_became_peered":"2024-09-17T08:32:12.545077+0000","last_unstale":"2024-09-17T08:32:38.441601+0000","last_undegraded":"2024-09-17T08:32:38.441601+0000","last_fullsized":"2024-09-17T08:32:38.441601+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:59.216859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"67'18","reported_seq":51,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439516+0000","last_change":"2024-09-17T08:32:08.522483+0000","last_active":"2024-09-17T08:32:38.439516+0000","last_peered":"2024-09-17T08:32:38.439516+0000","last_clean":"2024-09-17T08:32:38.439516+0000","last_became_active":"2024-09-17T08:32:08.522031+0000","last_became_peered":"2024-09-17T08:32:08.522031+0000","last_unstale":"2024-09-17T08:32:38.439516+0000","last_undegraded":"2024-09-17T08:32:38.439516+0000","last_fullsized":"2024-09-17T08:32:38.439516+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:37:58.206984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.440998+0000","last_change":"2024-09-17T08:32:06.518756+0000","last_active":"2024-09-17T08:32:38.440998+0000","last_peered":"2024-09-17T08:32:38.440998+0000","last_clean":"2024-09-17T08:32:38.440998+0000","last_became_active":"2024-09-17T08:32:06.518508+0000","last_became_peered":"2024-09-17T08:32:06.518508+0000","last_unstale":"2024-09-17T08:32:38.440998+0000","last_undegraded":"2024-09-17T08:32:38.440998+0000","last_fullsized":"2024-09-17T08:32:38.440998+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:50:54.216970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443253+0000","last_change":"2024-09-17T08:32:10.539361+0000","last_active":"2024-09-17T08:32:38.443253+0000","last_peered":"2024-09-17T08:32:38.443253+0000","last_clean":"2024-09-17T08:32:38.443253+0000","last_became_active":"2024-09-17T08:32:10.536390+0000","last_became_peered":"2024-09-17T08:32:10.536390+0000","last_unstale":"2024-09-17T08:32:38.443253+0000","last_undegraded":"2024-09-17T08:32:38.443253+0000","last_fullsized":"2024-09-17T08:32:38.443253+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:32:56.028375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443444+0000","last_change":"2024-09-17T08:32:12.549946+0000","last_active":"2024-09-17T08:32:38.443444+0000","last_peered":"2024-09-17T08:32:38.443444+0000","last_clean":"2024-09-17T08:32:38.443444+0000","last_became_active":"2024-09-17T08:32:12.548943+0000","last_became_peered":"2024-09-17T08:32:12.548943+0000","last_unstale":"2024-09-17T08:32:38.443444+0000","last_undegraded":"2024-09-17T08:32:38.443444+0000","last_fullsized":"2024-09-17T08:32:38.443444+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:47:00.317734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"67'14","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443393+0000","last_change":"2024-09-17T08:32:08.527353+0000","last_active":"2024-09-17T08:32:38.443393+0000","last_peered":"2024-09-17T08:32:38.443393+0000","last_clean":"2024-09-17T08:32:38.443393+0000","last_became_active":"2024-09-17T08:32:08.527074+0000","last_became_peered":"2024-09-17T08:32:08.527074+0000","last_unstale":"2024-09-17T08:32:38.443393+0000","last_undegraded":"2024-09-17T08:32:38.443393+0000","last_fullsized":"2024-09-17T08:32:38.443393+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:08.200483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441160+0000","last_change":"2024-09-17T08:32:06.515084+0000","last_active":"2024-09-17T08:32:38.441160+0000","last_peered":"2024-09-17T08:32:38.441160+0000","last_clean":"2024-09-17T08:32:38.441160+0000","last_became_active":"2024-09-17T08:32:06.514813+0000","last_became_peered":"2024-09-17T08:32:06.514813+0000","last_unstale":"2024-09-17T08:32:38.441160+0000","last_undegraded":"2024-09-17T08:32:38.441160+0000","last_fullsized":"2024-09-17T08:32:38.441160+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:29:11.823668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"67'8","reported_seq":43,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T08:33:29.923620+0000","last_change":"2024-09-17T08:32:10.535432+0000","last_active":"2024-09-17T08:33:29.923620+0000","last_peered":"2024-09-17T08:33:29.923620+0000","last_clean":"2024-09-17T08:33:29.923620+0000","last_became_active":"2024-09-17T08:32:10.535175+0000","last_became_peered":"2024-09-17T08:32:10.535175+0000","last_unstale":"2024-09-17T08:33:29.923620+0000","last_undegraded":"2024-09-17T08:33:29.923620+0000","last_fullsized":"2024-09-17T08:33:29.923620+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:42:15.467257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439696+0000","last_change":"2024-09-17T08:32:12.696753+0000","last_active":"2024-09-17T08:32:38.439696+0000","last_peered":"2024-09-17T08:32:38.439696+0000","last_clean":"2024-09-17T08:32:38.439696+0000","last_became_active":"2024-09-17T08:32:12.696495+0000","last_became_peered":"2024-09-17T08:32:12.696495+0000","last_unstale":"2024-09-17T08:32:38.439696+0000","last_undegraded":"2024-09-17T08:32:38.439696+0000","last_fullsized":"2024-09-17T08:32:38.439696+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:39:49.719496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"67'10","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.444401+0000","last_change":"2024-09-17T08:32:08.527446+0000","last_active":"2024-09-17T08:32:38.444401+0000","last_peered":"2024-09-17T08:32:38.444401+0000","last_clean":"2024-09-17T08:32:38.444401+0000","last_became_active":"2024-09-17T08:32:08.527129+0000","last_became_peered":"2024-09-17T08:32:08.527129+0000","last_unstale":"2024-09-17T08:32:38.444401+0000","last_undegraded":"2024-09-17T08:32:38.444401+0000","last_fullsized":"2024-09-17T08:32:38.444401+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:02:25.088242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441928+0000","last_change":"2024-09-17T08:32:06.520832+0000","last_active":"2024-09-17T08:32:38.441928+0000","last_peered":"2024-09-17T08:32:38.441928+0000","last_clean":"2024-09-17T08:32:38.441928+0000","last_became_active":"2024-09-17T08:32:06.520547+0000","last_became_peered":"2024-09-17T08:32:06.520547+0000","last_unstale":"2024-09-17T08:32:38.441928+0000","last_undegraded":"2024-09-17T08:32:38.441928+0000","last_fullsized":"2024-09-17T08:32:38.441928+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:07.463011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"67'8","reported_seq":43,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T08:33:29.923646+0000","last_change":"2024-09-17T08:32:10.543123+0000","last_active":"2024-09-17T08:33:29.923646+0000","last_peered":"2024-09-17T08:33:29.923646+0000","last_clean":"2024-09-17T08:33:29.923646+0000","last_became_active":"2024-09-17T08:32:10.542768+0000","last_became_peered":"2024-09-17T08:32:10.542768+0000","last_unstale":"2024-09-17T08:33:29.923646+0000","last_undegraded":"2024-09-17T08:33:29.923646+0000","last_fullsized":"2024-09-17T08:33:29.923646+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:11:46.728776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439470+0000","last_change":"2024-09-17T08:32:12.548678+0000","last_active":"2024-09-17T08:32:38.439470+0000","last_peered":"2024-09-17T08:32:38.439470+0000","last_clean":"2024-09-17T08:32:38.439470+0000","last_became_active":"2024-09-17T08:32:12.548412+0000","last_became_peered":"2024-09-17T08:32:12.548412+0000","last_unstale":"2024-09-17T08:32:38.439470+0000","last_undegraded":"2024-09-17T08:32:38.439470+0000","last_fullsized":"2024-09-17T08:32:38.439470+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:51.271230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"67'19","reported_seq":55,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.440453+0000","last_change":"2024-09-17T08:32:08.525359+0000","last_active":"2024-09-17T08:32:38.440453+0000","last_peered":"2024-09-17T08:32:38.440453+0000","last_clean":"2024-09-17T08:32:38.440453+0000","last_became_active":"2024-09-17T08:32:08.525089+0000","last_became_peered":"2024-09-17T08:32:08.525089+0000","last_unstale":"2024-09-17T08:32:38.440453+0000","last_undegraded":"2024-09-17T08:32:38.440453+0000","last_fullsized":"2024-09-17T08:32:38.440453+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:09.002769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.445662+0000","last_change":"2024-09-17T08:32:06.517321+0000","last_active":"2024-09-17T08:32:38.445662+0000","last_peered":"2024-09-17T08:32:38.445662+0000","last_clean":"2024-09-17T08:32:38.445662+0000","last_became_active":"2024-09-17T08:32:06.517061+0000","last_became_peered":"2024-09-17T08:32:06.517061+0000","last_unstale":"2024-09-17T08:32:38.445662+0000","last_undegraded":"2024-09-17T08:32:38.445662+0000","last_fullsized":"2024-09-17T08:32:38.445662+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:32:00.392092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.446466+0000","last_change":"2024-09-17T08:32:10.539083+0000","last_active":"2024-09-17T08:32:38.446466+0000","last_peered":"2024-09-17T08:32:38.446466+0000","last_clean":"2024-09-17T08:32:38.446466+0000","last_became_active":"2024-09-17T08:32:10.538761+0000","last_became_peered":"2024-09-17T08:32:10.538761+0000","last_unstale":"2024-09-17T08:32:38.446466+0000","last_undegraded":"2024-09-17T08:32:38.446466+0000","last_fullsized":"2024-09-17T08:32:38.446466+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:43:03.304139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.444423+0000","last_change":"2024-09-17T08:32:12.694826+0000","last_active":"2024-09-17T08:32:38.444423+0000","last_peered":"2024-09-17T08:32:38.444423+0000","last_clean":"2024-09-17T08:32:38.444423+0000","last_became_active":"2024-09-17T08:32:12.694275+0000","last_became_peered":"2024-09-17T08:32:12.694275+0000","last_unstale":"2024-09-17T08:32:38.444423+0000","last_undegraded":"2024-09-17T08:32:38.444423+0000","last_fullsized":"2024-09-17T08:32:38.444423+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:25:45.341443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"70'29","reported_seq":47,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T08:33:29.924038+0000","last_change":"2024-09-17T08:32:38.460715+0000","last_active":"2024-09-17T08:33:29.924038+0000","last_peered":"2024-09-17T08:33:29.924038+0000","last_clean":"2024-09-17T08:33:29.924038+0000","last_became_active":"2024-09-17T08:32:38.446928+0000","last_became_peered":"2024-09-17T08:32:38.446928+0000","last_unstale":"2024-09-17T08:33:29.924038+0000","last_undegraded":"2024-09-17T08:33:29.924038+0000","last_fullsized":"2024-09-17T08:33:29.924038+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:08:15.827269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443705+0000","last_change":"2024-09-17T08:32:06.514090+0000","last_active":"2024-09-17T08:32:38.443705+0000","last_peered":"2024-09-17T08:32:38.443705+0000","last_clean":"2024-09-17T08:32:38.443705+0000","last_became_active":"2024-09-17T08:32:06.513811+0000","last_became_peered":"2024-09-17T08:32:06.513811+0000","last_unstale":"2024-09-17T08:32:38.443705+0000","last_undegraded":"2024-09-17T08:32:38.443705+0000","last_fullsized":"2024-09-17T08:32:38.443705+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:24:06.372273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"64'2","reported_seq":28,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441528+0000","last_change":"2024-09-17T08:32:10.534037+0000","last_active":"2024-09-17T08:32:38.441528+0000","last_peered":"2024-09-17T08:32:38.441528+0000","last_clean":"2024-09-17T08:32:38.441528+0000","last_became_active":"2024-09-17T08:32:06.514988+0000","last_became_peered":"2024-09-17T08:32:06.514988+0000","last_unstale":"2024-09-17T08:32:38.441528+0000","last_undegraded":"2024-09-17T08:32:38.441528+0000","last_fullsized":"2024-09-17T08:32:38.441528+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:46.008515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020955869999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.440399+0000","last_change":"2024-09-17T08:32:10.546500+0000","last_active":"2024-09-17T08:32:38.440399+0000","last_peered":"2024-09-17T08:32:38.440399+0000","last_clean":"2024-09-17T08:32:38.440399+0000","last_became_active":"2024-09-17T08:32:10.546079+0000","last_became_peered":"2024-09-17T08:32:10.546079+0000","last_unstale":"2024-09-17T08:32:38.440399+0000","last_undegraded":"2024-09-17T08:32:38.440399+0000","last_fullsized":"2024-09-17T08:32:38.440399+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:43:17.414063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439728+0000","last_change":"2024-09-17T08:32:12.546332+0000","last_active":"2024-09-17T08:32:38.439728+0000","last_peered":"2024-09-17T08:32:38.439728+0000","last_clean":"2024-09-17T08:32:38.439728+0000","last_became_active":"2024-09-17T08:32:12.545993+0000","last_became_peered":"2024-09-17T08:32:12.545993+0000","last_unstale":"2024-09-17T08:32:38.439728+0000","last_undegraded":"2024-09-17T08:32:38.439728+0000","last_fullsized":"2024-09-17T08:32:38.439728+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:11:00.528909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"67'13","reported_seq":46,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.446141+0000","last_change":"2024-09-17T08:32:08.527426+0000","last_active":"2024-09-17T08:32:38.446141+0000","last_peered":"2024-09-17T08:32:38.446141+0000","last_clean":"2024-09-17T08:32:38.446141+0000","last_became_active":"2024-09-17T08:32:08.527030+0000","last_became_peered":"2024-09-17T08:32:08.527030+0000","last_unstale":"2024-09-17T08:32:38.446141+0000","last_undegraded":"2024-09-17T08:32:38.446141+0000","last_fullsized":"2024-09-17T08:32:38.446141+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:06:54.129541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"60'1","reported_seq":25,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.446676+0000","last_change":"2024-09-17T08:32:06.515617+0000","last_active":"2024-09-17T08:32:38.446676+0000","last_peered":"2024-09-17T08:32:38.446676+0000","last_clean":"2024-09-17T08:32:38.446676+0000","last_became_active":"2024-09-17T08:32:06.515325+0000","last_became_peered":"2024-09-17T08:32:06.515325+0000","last_unstale":"2024-09-17T08:32:38.446676+0000","last_undegraded":"2024-09-17T08:32:38.446676+0000","last_fullsized":"2024-09-17T08:32:38.446676+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:01:22.001817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"62'1","reported_seq":27,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443735+0000","last_change":"2024-09-17T08:32:10.528937+0000","last_active":"2024-09-17T08:32:38.443735+0000","last_peered":"2024-09-17T08:32:38.443735+0000","last_clean":"2024-09-17T08:32:38.443735+0000","last_became_active":"2024-09-17T08:32:06.512453+0000","last_became_peered":"2024-09-17T08:32:06.512453+0000","last_unstale":"2024-09-17T08:32:38.443735+0000","last_undegraded":"2024-09-17T08:32:38.443735+0000","last_fullsized":"2024-09-17T08:32:38.443735+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:49:16.482240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042636900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.444475+0000","last_change":"2024-09-17T08:32:10.536503+0000","last_active":"2024-09-17T08:32:38.444475+0000","last_peered":"2024-09-17T08:32:38.444475+0000","last_clean":"2024-09-17T08:32:38.444475+0000","last_became_active":"2024-09-17T08:32:10.536201+0000","last_became_peered":"2024-09-17T08:32:10.536201+0000","last_unstale":"2024-09-17T08:32:38.444475+0000","last_undegraded":"2024-09-17T08:32:38.444475+0000","last_fullsized":"2024-09-17T08:32:38.444475+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:35:41.890464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439686+0000","last_change":"2024-09-17T08:32:12.694472+0000","last_active":"2024-09-17T08:32:38.439686+0000","last_peered":"2024-09-17T08:32:38.439686+0000","last_clean":"2024-09-17T08:32:38.439686+0000","last_became_active":"2024-09-17T08:32:12.694188+0000","last_became_peered":"2024-09-17T08:32:12.694188+0000","last_unstale":"2024-09-17T08:32:38.439686+0000","last_undegraded":"2024-09-17T08:32:38.439686+0000","last_fullsized":"2024-09-17T08:32:38.439686+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:48:38.549255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"67'12","reported_seq":37,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441075+0000","last_change":"2024-09-17T08:32:08.526555+0000","last_active":"2024-09-17T08:32:38.441075+0000","last_peered":"2024-09-17T08:32:38.441075+0000","last_clean":"2024-09-17T08:32:38.441075+0000","last_became_active":"2024-09-17T08:32:08.526232+0000","last_became_peered":"2024-09-17T08:32:08.526232+0000","last_unstale":"2024-09-17T08:32:38.441075+0000","last_undegraded":"2024-09-17T08:32:38.441075+0000","last_fullsized":"2024-09-17T08:32:38.441075+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:31:46.911842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439985+0000","last_change":"2024-09-17T08:32:06.519792+0000","last_active":"2024-09-17T08:32:38.439985+0000","last_peered":"2024-09-17T08:32:38.439985+0000","last_clean":"2024-09-17T08:32:38.439985+0000","last_became_active":"2024-09-17T08:32:06.519537+0000","last_became_peered":"2024-09-17T08:32:06.519537+0000","last_unstale":"2024-09-17T08:32:38.439985+0000","last_undegraded":"2024-09-17T08:32:38.439985+0000","last_fullsized":"2024-09-17T08:32:38.439985+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:43:26.888478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"68'5","reported_seq":111,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T08:33:33.286407+0000","last_change":"2024-09-17T08:32:10.532085+0000","last_active":"2024-09-17T08:33:33.286407+0000","last_peered":"2024-09-17T08:33:33.286407+0000","last_clean":"2024-09-17T08:33:33.286407+0000","last_became_active":"2024-09-17T08:32:06.520521+0000","last_became_peered":"2024-09-17T08:32:06.520521+0000","last_unstale":"2024-09-17T08:33:33.286407+0000","last_undegraded":"2024-09-17T08:33:33.286407+0000","last_fullsized":"2024-09-17T08:33:33.286407+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:04:42.051173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000319551,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":82,"num_read_kb":77,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.442067+0000","last_change":"2024-09-17T08:32:10.540716+0000","last_active":"2024-09-17T08:32:38.442067+0000","last_peered":"2024-09-17T08:32:38.442067+0000","last_clean":"2024-09-17T08:32:38.442067+0000","last_became_active":"2024-09-17T08:32:10.540289+0000","last_became_peered":"2024-09-17T08:32:10.540289+0000","last_unstale":"2024-09-17T08:32:38.442067+0000","last_undegraded":"2024-09-17T08:32:38.442067+0000","last_fullsized":"2024-09-17T08:32:38.442067+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:23:35.377736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.445271+0000","last_change":"2024-09-17T08:32:12.550135+0000","last_active":"2024-09-17T08:32:38.445271+0000","last_peered":"2024-09-17T08:32:38.445271+0000","last_clean":"2024-09-17T08:32:38.445271+0000","last_became_active":"2024-09-17T08:32:12.549831+0000","last_became_peered":"2024-09-17T08:32:12.549831+0000","last_unstale":"2024-09-17T08:32:38.445271+0000","last_undegraded":"2024-09-17T08:32:38.445271+0000","last_fullsized":"2024-09-17T08:32:38.445271+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:30:31.912823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"67'14","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.446309+0000","last_change":"2024-09-17T08:32:08.525573+0000","last_active":"2024-09-17T08:32:38.446309+0000","last_peered":"2024-09-17T08:32:38.446309+0000","last_clean":"2024-09-17T08:32:38.446309+0000","last_became_active":"2024-09-17T08:32:08.525277+0000","last_became_peered":"2024-09-17T08:32:08.525277+0000","last_unstale":"2024-09-17T08:32:38.446309+0000","last_undegraded":"2024-09-17T08:32:38.446309+0000","last_fullsized":"2024-09-17T08:32:38.446309+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:39:38.647697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439991+0000","last_change":"2024-09-17T08:32:06.520661+0000","last_active":"2024-09-17T08:32:38.439991+0000","last_peered":"2024-09-17T08:32:38.439991+0000","last_clean":"2024-09-17T08:32:38.439991+0000","last_became_active":"2024-09-17T08:32:06.520414+0000","last_became_peered":"2024-09-17T08:32:06.520414+0000","last_unstale":"2024-09-17T08:32:38.439991+0000","last_undegraded":"2024-09-17T08:32:38.439991+0000","last_fullsized":"2024-09-17T08:32:38.439991+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:47:17.219305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"71'128","reported_seq":266,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T08:32:46.105522+0000","last_change":"2024-09-17T08:31:56.830022+0000","last_active":"2024-09-17T08:32:46.105522+0000","last_peered":"2024-09-17T08:32:46.105522+0000","last_clean":"2024-09-17T08:32:46.105522+0000","last_became_active":"2024-09-17T08:31:56.829586+0000","last_became_peered":"2024-09-17T08:31:56.829586+0000","last_unstale":"2024-09-17T08:32:46.105522+0000","last_undegraded":"2024-09-17T08:32:46.105522+0000","last_fullsized":"2024-09-17T08:32:46.105522+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:29:53.628973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:29:53.628973+0000","last_clean_scrub_stamp":"2024-09-17T08:29:53.628973+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:27:27.657455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439614+0000","last_change":"2024-09-17T08:32:10.537678+0000","last_active":"2024-09-17T08:32:38.439614+0000","last_peered":"2024-09-17T08:32:38.439614+0000","last_clean":"2024-09-17T08:32:38.439614+0000","last_became_active":"2024-09-17T08:32:10.537387+0000","last_became_peered":"2024-09-17T08:32:10.537387+0000","last_unstale":"2024-09-17T08:32:38.439614+0000","last_undegraded":"2024-09-17T08:32:38.439614+0000","last_fullsized":"2024-09-17T08:32:38.439614+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:03:35.883626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.442013+0000","last_change":"2024-09-17T08:32:12.541915+0000","last_active":"2024-09-17T08:32:38.442013+0000","last_peered":"2024-09-17T08:32:38.442013+0000","last_clean":"2024-09-17T08:32:38.442013+0000","last_became_active":"2024-09-17T08:32:12.541620+0000","last_became_peered":"2024-09-17T08:32:12.541620+0000","last_unstale":"2024-09-17T08:32:38.442013+0000","last_undegraded":"2024-09-17T08:32:38.442013+0000","last_fullsized":"2024-09-17T08:32:38.442013+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:42:21.610221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"67'11","reported_seq":59,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.453255+0000","last_change":"2024-09-17T08:32:38.453025+0000","last_active":"2024-09-17T08:32:38.453255+0000","last_peered":"2024-09-17T08:32:38.453255+0000","last_clean":"2024-09-17T08:32:38.453255+0000","last_became_active":"2024-09-17T08:32:38.447402+0000","last_became_peered":"2024-09-17T08:32:38.447402+0000","last_unstale":"2024-09-17T08:32:38.453255+0000","last_undegraded":"2024-09-17T08:32:38.453255+0000","last_fullsized":"2024-09-17T08:32:38.453255+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:44:36.541038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443596+0000","last_change":"2024-09-17T08:32:06.514823+0000","last_active":"2024-09-17T08:32:38.443596+0000","last_peered":"2024-09-17T08:32:38.443596+0000","last_clean":"2024-09-17T08:32:38.443596+0000","last_became_active":"2024-09-17T08:32:06.514130+0000","last_became_peered":"2024-09-17T08:32:06.514130+0000","last_unstale":"2024-09-17T08:32:38.443596+0000","last_undegraded":"2024-09-17T08:32:38.443596+0000","last_fullsized":"2024-09-17T08:32:38.443596+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:25:57.872879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441834+0000","last_change":"2024-09-17T08:32:10.542601+0000","last_active":"2024-09-17T08:32:38.441834+0000","last_peered":"2024-09-17T08:32:38.441834+0000","last_clean":"2024-09-17T08:32:38.441834+0000","last_became_active":"2024-09-17T08:32:10.542295+0000","last_became_peered":"2024-09-17T08:32:10.542295+0000","last_unstale":"2024-09-17T08:32:38.441834+0000","last_undegraded":"2024-09-17T08:32:38.441834+0000","last_fullsized":"2024-09-17T08:32:38.441834+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:06:48.334079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439596+0000","last_change":"2024-09-17T08:32:12.694578+0000","last_active":"2024-09-17T08:32:38.439596+0000","last_peered":"2024-09-17T08:32:38.439596+0000","last_clean":"2024-09-17T08:32:38.439596+0000","last_became_active":"2024-09-17T08:32:12.694275+0000","last_became_peered":"2024-09-17T08:32:12.694275+0000","last_unstale":"2024-09-17T08:32:38.439596+0000","last_undegraded":"2024-09-17T08:32:38.439596+0000","last_fullsized":"2024-09-17T08:32:38.439596+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:57:22.633148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"67'17","reported_seq":74,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.467792+0000","last_change":"2024-09-17T08:32:38.467605+0000","last_active":"2024-09-17T08:32:38.467792+0000","last_peered":"2024-09-17T08:32:38.467792+0000","last_clean":"2024-09-17T08:32:38.467792+0000","last_became_active":"2024-09-17T08:32:38.447388+0000","last_became_peered":"2024-09-17T08:32:38.447388+0000","last_unstale":"2024-09-17T08:32:38.467792+0000","last_undegraded":"2024-09-17T08:32:38.467792+0000","last_fullsized":"2024-09-17T08:32:38.467792+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:16.348652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.446798+0000","last_change":"2024-09-17T08:32:06.515756+0000","last_active":"2024-09-17T08:32:38.446798+0000","last_peered":"2024-09-17T08:32:38.446798+0000","last_clean":"2024-09-17T08:32:38.446798+0000","last_became_active":"2024-09-17T08:32:06.515481+0000","last_became_peered":"2024-09-17T08:32:06.515481+0000","last_unstale":"2024-09-17T08:32:38.446798+0000","last_undegraded":"2024-09-17T08:32:38.446798+0000","last_fullsized":"2024-09-17T08:32:38.446798+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:04:33.289415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.445365+0000","last_change":"2024-09-17T08:32:10.546682+0000","last_active":"2024-09-17T08:32:38.445365+0000","last_peered":"2024-09-17T08:32:38.445365+0000","last_clean":"2024-09-17T08:32:38.445365+0000","last_became_active":"2024-09-17T08:32:10.546132+0000","last_became_peered":"2024-09-17T08:32:10.546132+0000","last_unstale":"2024-09-17T08:32:38.445365+0000","last_undegraded":"2024-09-17T08:32:38.445365+0000","last_fullsized":"2024-09-17T08:32:38.445365+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:28:01.966126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.444691+0000","last_change":"2024-09-17T08:32:12.546672+0000","last_active":"2024-09-17T08:32:38.444691+0000","last_peered":"2024-09-17T08:32:38.444691+0000","last_clean":"2024-09-17T08:32:38.444691+0000","last_became_active":"2024-09-17T08:32:12.546417+0000","last_became_peered":"2024-09-17T08:32:12.546417+0000","last_unstale":"2024-09-17T08:32:38.444691+0000","last_undegraded":"2024-09-17T08:32:38.444691+0000","last_fullsized":"2024-09-17T08:32:38.444691+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:20:23.859665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"67'10","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443674+0000","last_change":"2024-09-17T08:32:08.533942+0000","last_active":"2024-09-17T08:32:38.443674+0000","last_peered":"2024-09-17T08:32:38.443674+0000","last_clean":"2024-09-17T08:32:38.443674+0000","last_became_active":"2024-09-17T08:32:08.533664+0000","last_became_peered":"2024-09-17T08:32:08.533664+0000","last_unstale":"2024-09-17T08:32:38.443674+0000","last_undegraded":"2024-09-17T08:32:38.443674+0000","last_fullsized":"2024-09-17T08:32:38.443674+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:43:57.996920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441013+0000","last_change":"2024-09-17T08:32:06.512628+0000","last_active":"2024-09-17T08:32:38.441013+0000","last_peered":"2024-09-17T08:32:38.441013+0000","last_clean":"2024-09-17T08:32:38.441013+0000","last_became_active":"2024-09-17T08:32:06.512049+0000","last_became_peered":"2024-09-17T08:32:06.512049+0000","last_unstale":"2024-09-17T08:32:38.441013+0000","last_undegraded":"2024-09-17T08:32:38.441013+0000","last_fullsized":"2024-09-17T08:32:38.441013+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:43:37.306297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"67'8","reported_seq":43,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T08:33:29.923646+0000","last_change":"2024-09-17T08:32:10.536769+0000","last_active":"2024-09-17T08:33:29.923646+0000","last_peered":"2024-09-17T08:33:29.923646+0000","last_clean":"2024-09-17T08:33:29.923646+0000","last_became_active":"2024-09-17T08:32:10.536216+0000","last_became_peered":"2024-09-17T08:32:10.536216+0000","last_unstale":"2024-09-17T08:33:29.923646+0000","last_undegraded":"2024-09-17T08:33:29.923646+0000","last_fullsized":"2024-09-17T08:33:29.923646+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:17:57.131915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443764+0000","last_change":"2024-09-17T08:32:12.549748+0000","last_active":"2024-09-17T08:32:38.443764+0000","last_peered":"2024-09-17T08:32:38.443764+0000","last_clean":"2024-09-17T08:32:38.443764+0000","last_became_active":"2024-09-17T08:32:12.549429+0000","last_became_peered":"2024-09-17T08:32:12.549429+0000","last_unstale":"2024-09-17T08:32:38.443764+0000","last_undegraded":"2024-09-17T08:32:38.443764+0000","last_fullsized":"2024-09-17T08:32:38.443764+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:57:32.749218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"67'9","reported_seq":35,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.440367+0000","last_change":"2024-09-17T08:32:08.527343+0000","last_active":"2024-09-17T08:32:38.440367+0000","last_peered":"2024-09-17T08:32:38.440367+0000","last_clean":"2024-09-17T08:32:38.440367+0000","last_became_active":"2024-09-17T08:32:08.527011+0000","last_became_peered":"2024-09-17T08:32:08.527011+0000","last_unstale":"2024-09-17T08:32:38.440367+0000","last_undegraded":"2024-09-17T08:32:38.440367+0000","last_fullsized":"2024-09-17T08:32:38.440367+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:19:36.249578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.442223+0000","last_change":"2024-09-17T08:32:06.521196+0000","last_active":"2024-09-17T08:32:38.442223+0000","last_peered":"2024-09-17T08:32:38.442223+0000","last_clean":"2024-09-17T08:32:38.442223+0000","last_became_active":"2024-09-17T08:32:06.520907+0000","last_became_peered":"2024-09-17T08:32:06.520907+0000","last_unstale":"2024-09-17T08:32:38.442223+0000","last_undegraded":"2024-09-17T08:32:38.442223+0000","last_fullsized":"2024-09-17T08:32:38.442223+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:32:54.848677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443864+0000","last_change":"2024-09-17T08:32:10.537279+0000","last_active":"2024-09-17T08:32:38.443864+0000","last_peered":"2024-09-17T08:32:38.443864+0000","last_clean":"2024-09-17T08:32:38.443864+0000","last_became_active":"2024-09-17T08:32:10.537001+0000","last_became_peered":"2024-09-17T08:32:10.537001+0000","last_unstale":"2024-09-17T08:32:38.443864+0000","last_undegraded":"2024-09-17T08:32:38.443864+0000","last_fullsized":"2024-09-17T08:32:38.443864+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:16:12.797016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439903+0000","last_change":"2024-09-17T08:32:12.549575+0000","last_active":"2024-09-17T08:32:38.439903+0000","last_peered":"2024-09-17T08:32:38.439903+0000","last_clean":"2024-09-17T08:32:38.439903+0000","last_became_active":"2024-09-17T08:32:12.549193+0000","last_became_peered":"2024-09-17T08:32:12.549193+0000","last_unstale":"2024-09-17T08:32:38.439903+0000","last_undegraded":"2024-09-17T08:32:38.439903+0000","last_fullsized":"2024-09-17T08:32:38.439903+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:59:49.773354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"67'19","reported_seq":50,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.446615+0000","last_change":"2024-09-17T08:32:08.528428+0000","last_active":"2024-09-17T08:32:38.446615+0000","last_peered":"2024-09-17T08:32:38.446615+0000","last_clean":"2024-09-17T08:32:38.446615+0000","last_became_active":"2024-09-17T08:32:08.528192+0000","last_became_peered":"2024-09-17T08:32:08.528192+0000","last_unstale":"2024-09-17T08:32:38.446615+0000","last_undegraded":"2024-09-17T08:32:38.446615+0000","last_fullsized":"2024-09-17T08:32:38.446615+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:01.049301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439890+0000","last_change":"2024-09-17T08:32:06.517665+0000","last_active":"2024-09-17T08:32:38.439890+0000","last_peered":"2024-09-17T08:32:38.439890+0000","last_clean":"2024-09-17T08:32:38.439890+0000","last_became_active":"2024-09-17T08:32:06.517407+0000","last_became_peered":"2024-09-17T08:32:06.517407+0000","last_unstale":"2024-09-17T08:32:38.439890+0000","last_undegraded":"2024-09-17T08:32:38.439890+0000","last_fullsized":"2024-09-17T08:32:38.439890+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:58:13.456366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.444072+0000","last_change":"2024-09-17T08:32:10.541218+0000","last_active":"2024-09-17T08:32:38.444072+0000","last_peered":"2024-09-17T08:32:38.444072+0000","last_clean":"2024-09-17T08:32:38.444072+0000","last_became_active":"2024-09-17T08:32:10.540908+0000","last_became_peered":"2024-09-17T08:32:10.540908+0000","last_unstale":"2024-09-17T08:32:38.444072+0000","last_undegraded":"2024-09-17T08:32:38.444072+0000","last_fullsized":"2024-09-17T08:32:38.444072+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:51.618576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439795+0000","last_change":"2024-09-17T08:32:12.550666+0000","last_active":"2024-09-17T08:32:38.439795+0000","last_peered":"2024-09-17T08:32:38.439795+0000","last_clean":"2024-09-17T08:32:38.439795+0000","last_became_active":"2024-09-17T08:32:12.550356+0000","last_became_peered":"2024-09-17T08:32:12.550356+0000","last_unstale":"2024-09-17T08:32:38.439795+0000","last_undegraded":"2024-09-17T08:32:38.439795+0000","last_fullsized":"2024-09-17T08:32:38.439795+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:31:18.051423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"67'12","reported_seq":42,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443489+0000","last_change":"2024-09-17T08:32:08.533629+0000","last_active":"2024-09-17T08:32:38.443489+0000","last_peered":"2024-09-17T08:32:38.443489+0000","last_clean":"2024-09-17T08:32:38.443489+0000","last_became_active":"2024-09-17T08:32:08.533303+0000","last_became_peered":"2024-09-17T08:32:08.533303+0000","last_unstale":"2024-09-17T08:32:38.443489+0000","last_undegraded":"2024-09-17T08:32:38.443489+0000","last_fullsized":"2024-09-17T08:32:38.443489+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:08:05.067425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439968+0000","last_change":"2024-09-17T08:32:06.518413+0000","last_active":"2024-09-17T08:32:38.439968+0000","last_peered":"2024-09-17T08:32:38.439968+0000","last_clean":"2024-09-17T08:32:38.439968+0000","last_became_active":"2024-09-17T08:32:06.518073+0000","last_became_peered":"2024-09-17T08:32:06.518073+0000","last_unstale":"2024-09-17T08:32:38.439968+0000","last_undegraded":"2024-09-17T08:32:38.439968+0000","last_fullsized":"2024-09-17T08:32:38.439968+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:07:08.022690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443416+0000","last_change":"2024-09-17T08:32:10.540363+0000","last_active":"2024-09-17T08:32:38.443416+0000","last_peered":"2024-09-17T08:32:38.443416+0000","last_clean":"2024-09-17T08:32:38.443416+0000","last_became_active":"2024-09-17T08:32:10.540055+0000","last_became_peered":"2024-09-17T08:32:10.540055+0000","last_unstale":"2024-09-17T08:32:38.443416+0000","last_undegraded":"2024-09-17T08:32:38.443416+0000","last_fullsized":"2024-09-17T08:32:38.443416+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:31:37.966877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441143+0000","last_change":"2024-09-17T08:32:12.548919+0000","last_active":"2024-09-17T08:32:38.441143+0000","last_peered":"2024-09-17T08:32:38.441143+0000","last_clean":"2024-09-17T08:32:38.441143+0000","last_became_active":"2024-09-17T08:32:12.546326+0000","last_became_peered":"2024-09-17T08:32:12.546326+0000","last_unstale":"2024-09-17T08:32:38.441143+0000","last_undegraded":"2024-09-17T08:32:38.441143+0000","last_fullsized":"2024-09-17T08:32:38.441143+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:25.361371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"67'15","reported_seq":44,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441318+0000","last_change":"2024-09-17T08:32:08.526395+0000","last_active":"2024-09-17T08:32:38.441318+0000","last_peered":"2024-09-17T08:32:38.441318+0000","last_clean":"2024-09-17T08:32:38.441318+0000","last_became_active":"2024-09-17T08:32:08.526118+0000","last_became_peered":"2024-09-17T08:32:08.526118+0000","last_unstale":"2024-09-17T08:32:38.441318+0000","last_undegraded":"2024-09-17T08:32:38.441318+0000","last_fullsized":"2024-09-17T08:32:38.441318+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:22:45.111550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"60'2","reported_seq":28,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439995+0000","last_change":"2024-09-17T08:32:06.519365+0000","last_active":"2024-09-17T08:32:38.439995+0000","last_peered":"2024-09-17T08:32:38.439995+0000","last_clean":"2024-09-17T08:32:38.439995+0000","last_became_active":"2024-09-17T08:32:06.519106+0000","last_became_peered":"2024-09-17T08:32:06.519106+0000","last_unstale":"2024-09-17T08:32:38.439995+0000","last_undegraded":"2024-09-17T08:32:38.439995+0000","last_fullsized":"2024-09-17T08:32:38.439995+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:15:50.606571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"67'8","reported_seq":43,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T08:33:29.923745+0000","last_change":"2024-09-17T08:32:10.544235+0000","last_active":"2024-09-17T08:33:29.923745+0000","last_peered":"2024-09-17T08:33:29.923745+0000","last_clean":"2024-09-17T08:33:29.923745+0000","last_became_active":"2024-09-17T08:32:10.543872+0000","last_became_peered":"2024-09-17T08:32:10.543872+0000","last_unstale":"2024-09-17T08:33:29.923745+0000","last_undegraded":"2024-09-17T08:33:29.923745+0000","last_fullsized":"2024-09-17T08:33:29.923745+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:47.645731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441372+0000","last_change":"2024-09-17T08:32:12.695219+0000","last_active":"2024-09-17T08:32:38.441372+0000","last_peered":"2024-09-17T08:32:38.441372+0000","last_clean":"2024-09-17T08:32:38.441372+0000","last_became_active":"2024-09-17T08:32:12.694097+0000","last_became_peered":"2024-09-17T08:32:12.694097+0000","last_unstale":"2024-09-17T08:32:38.441372+0000","last_undegraded":"2024-09-17T08:32:38.441372+0000","last_fullsized":"2024-09-17T08:32:38.441372+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:11:20.311913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.445787+0000","last_change":"2024-09-17T08:32:06.516655+0000","last_active":"2024-09-17T08:32:38.445787+0000","last_peered":"2024-09-17T08:32:38.445787+0000","last_clean":"2024-09-17T08:32:38.445787+0000","last_became_active":"2024-09-17T08:32:06.516375+0000","last_became_peered":"2024-09-17T08:32:06.516375+0000","last_unstale":"2024-09-17T08:32:38.445787+0000","last_undegraded":"2024-09-17T08:32:38.445787+0000","last_fullsized":"2024-09-17T08:32:38.445787+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:58:03.022084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"67'6","reported_seq":28,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.440848+0000","last_change":"2024-09-17T08:32:08.532685+0000","last_active":"2024-09-17T08:32:38.440848+0000","last_peered":"2024-09-17T08:32:38.440848+0000","last_clean":"2024-09-17T08:32:38.440848+0000","last_became_active":"2024-09-17T08:32:08.532410+0000","last_became_peered":"2024-09-17T08:32:08.532410+0000","last_unstale":"2024-09-17T08:32:38.440848+0000","last_undegraded":"2024-09-17T08:32:38.440848+0000","last_fullsized":"2024-09-17T08:32:38.440848+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:37:42.638603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441758+0000","last_change":"2024-09-17T08:32:10.537113+0000","last_active":"2024-09-17T08:32:38.441758+0000","last_peered":"2024-09-17T08:32:38.441758+0000","last_clean":"2024-09-17T08:32:38.441758+0000","last_became_active":"2024-09-17T08:32:10.536713+0000","last_became_peered":"2024-09-17T08:32:10.536713+0000","last_unstale":"2024-09-17T08:32:38.441758+0000","last_undegraded":"2024-09-17T08:32:38.441758+0000","last_fullsized":"2024-09-17T08:32:38.441758+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:14.257796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439605+0000","last_change":"2024-09-17T08:32:12.695165+0000","last_active":"2024-09-17T08:32:38.439605+0000","last_peered":"2024-09-17T08:32:38.439605+0000","last_clean":"2024-09-17T08:32:38.439605+0000","last_became_active":"2024-09-17T08:32:12.694913+0000","last_became_peered":"2024-09-17T08:32:12.694913+0000","last_unstale":"2024-09-17T08:32:38.439605+0000","last_undegraded":"2024-09-17T08:32:38.439605+0000","last_fullsized":"2024-09-17T08:32:38.439605+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:11:34.354636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"67'9","reported_seq":35,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441114+0000","last_change":"2024-09-17T08:32:08.523545+0000","last_active":"2024-09-17T08:32:38.441114+0000","last_peered":"2024-09-17T08:32:38.441114+0000","last_clean":"2024-09-17T08:32:38.441114+0000","last_became_active":"2024-09-17T08:32:08.523212+0000","last_became_peered":"2024-09-17T08:32:08.523212+0000","last_unstale":"2024-09-17T08:32:38.441114+0000","last_undegraded":"2024-09-17T08:32:38.441114+0000","last_fullsized":"2024-09-17T08:32:38.441114+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:52:42.189935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439547+0000","last_change":"2024-09-17T08:32:06.518457+0000","last_active":"2024-09-17T08:32:38.439547+0000","last_peered":"2024-09-17T08:32:38.439547+0000","last_clean":"2024-09-17T08:32:38.439547+0000","last_became_active":"2024-09-17T08:32:06.518126+0000","last_became_peered":"2024-09-17T08:32:06.518126+0000","last_unstale":"2024-09-17T08:32:38.439547+0000","last_undegraded":"2024-09-17T08:32:38.439547+0000","last_fullsized":"2024-09-17T08:32:38.439547+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:35:51.082723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.440693+0000","last_change":"2024-09-17T08:32:10.536866+0000","last_active":"2024-09-17T08:32:38.440693+0000","last_peered":"2024-09-17T08:32:38.440693+0000","last_clean":"2024-09-17T08:32:38.440693+0000","last_became_active":"2024-09-17T08:32:10.536586+0000","last_became_peered":"2024-09-17T08:32:10.536586+0000","last_unstale":"2024-09-17T08:32:38.440693+0000","last_undegraded":"2024-09-17T08:32:38.440693+0000","last_fullsized":"2024-09-17T08:32:38.440693+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:20:34.975150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443422+0000","last_change":"2024-09-17T08:32:12.552715+0000","last_active":"2024-09-17T08:32:38.443422+0000","last_peered":"2024-09-17T08:32:38.443422+0000","last_clean":"2024-09-17T08:32:38.443422+0000","last_became_active":"2024-09-17T08:32:12.552430+0000","last_became_peered":"2024-09-17T08:32:12.552430+0000","last_unstale":"2024-09-17T08:32:38.443422+0000","last_undegraded":"2024-09-17T08:32:38.443422+0000","last_fullsized":"2024-09-17T08:32:38.443422+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:43.203832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"67'9","reported_seq":35,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441793+0000","last_change":"2024-09-17T08:32:08.531006+0000","last_active":"2024-09-17T08:32:38.441793+0000","last_peered":"2024-09-17T08:32:38.441793+0000","last_clean":"2024-09-17T08:32:38.441793+0000","last_became_active":"2024-09-17T08:32:08.530746+0000","last_became_peered":"2024-09-17T08:32:08.530746+0000","last_unstale":"2024-09-17T08:32:38.441793+0000","last_undegraded":"2024-09-17T08:32:38.441793+0000","last_fullsized":"2024-09-17T08:32:38.441793+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:11:38.326764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.440052+0000","last_change":"2024-09-17T08:32:06.521875+0000","last_active":"2024-09-17T08:32:38.440052+0000","last_peered":"2024-09-17T08:32:38.440052+0000","last_clean":"2024-09-17T08:32:38.440052+0000","last_became_active":"2024-09-17T08:32:06.521586+0000","last_became_peered":"2024-09-17T08:32:06.521586+0000","last_unstale":"2024-09-17T08:32:38.440052+0000","last_undegraded":"2024-09-17T08:32:38.440052+0000","last_fullsized":"2024-09-17T08:32:38.440052+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:32:15.981385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"67'8","reported_seq":46,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T08:33:29.923874+0000","last_change":"2024-09-17T08:32:10.533118+0000","last_active":"2024-09-17T08:33:29.923874+0000","last_peered":"2024-09-17T08:33:29.923874+0000","last_clean":"2024-09-17T08:33:29.923874+0000","last_became_active":"2024-09-17T08:32:10.532849+0000","last_became_peered":"2024-09-17T08:32:10.532849+0000","last_unstale":"2024-09-17T08:33:29.923874+0000","last_undegraded":"2024-09-17T08:33:29.923874+0000","last_fullsized":"2024-09-17T08:33:29.923874+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:12:16.625685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443986+0000","last_change":"2024-09-17T08:32:12.549406+0000","last_active":"2024-09-17T08:32:38.443986+0000","last_peered":"2024-09-17T08:32:38.443986+0000","last_clean":"2024-09-17T08:32:38.443986+0000","last_became_active":"2024-09-17T08:32:12.548994+0000","last_became_peered":"2024-09-17T08:32:12.548994+0000","last_unstale":"2024-09-17T08:32:38.443986+0000","last_undegraded":"2024-09-17T08:32:38.443986+0000","last_fullsized":"2024-09-17T08:32:38.443986+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:06.757239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"67'10","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441244+0000","last_change":"2024-09-17T08:32:08.531863+0000","last_active":"2024-09-17T08:32:38.441244+0000","last_peered":"2024-09-17T08:32:38.441244+0000","last_clean":"2024-09-17T08:32:38.441244+0000","last_became_active":"2024-09-17T08:32:08.531586+0000","last_became_peered":"2024-09-17T08:32:08.531586+0000","last_unstale":"2024-09-17T08:32:38.441244+0000","last_undegraded":"2024-09-17T08:32:38.441244+0000","last_fullsized":"2024-09-17T08:32:38.441244+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:41:23.793510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439874+0000","last_change":"2024-09-17T08:32:06.518456+0000","last_active":"2024-09-17T08:32:38.439874+0000","last_peered":"2024-09-17T08:32:38.439874+0000","last_clean":"2024-09-17T08:32:38.439874+0000","last_became_active":"2024-09-17T08:32:06.518097+0000","last_became_peered":"2024-09-17T08:32:06.518097+0000","last_unstale":"2024-09-17T08:32:38.439874+0000","last_undegraded":"2024-09-17T08:32:38.439874+0000","last_fullsized":"2024-09-17T08:32:38.439874+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:06.171179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"67'8","reported_seq":43,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T08:33:29.923748+0000","last_change":"2024-09-17T08:32:10.540578+0000","last_active":"2024-09-17T08:33:29.923748+0000","last_peered":"2024-09-17T08:33:29.923748+0000","last_clean":"2024-09-17T08:33:29.923748+0000","last_became_active":"2024-09-17T08:32:10.540225+0000","last_became_peered":"2024-09-17T08:32:10.540225+0000","last_unstale":"2024-09-17T08:33:29.923748+0000","last_undegraded":"2024-09-17T08:33:29.923748+0000","last_fullsized":"2024-09-17T08:33:29.923748+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:03:26.564210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.440862+0000","last_change":"2024-09-17T08:32:12.548390+0000","last_active":"2024-09-17T08:32:38.440862+0000","last_peered":"2024-09-17T08:32:38.440862+0000","last_clean":"2024-09-17T08:32:38.440862+0000","last_became_active":"2024-09-17T08:32:12.547813+0000","last_became_peered":"2024-09-17T08:32:12.547813+0000","last_unstale":"2024-09-17T08:32:38.440862+0000","last_undegraded":"2024-09-17T08:32:38.440862+0000","last_fullsized":"2024-09-17T08:32:38.440862+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:46.939265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"67'11","reported_seq":38,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443524+0000","last_change":"2024-09-17T08:32:08.530137+0000","last_active":"2024-09-17T08:32:38.443524+0000","last_peered":"2024-09-17T08:32:38.443524+0000","last_clean":"2024-09-17T08:32:38.443524+0000","last_became_active":"2024-09-17T08:32:08.529908+0000","last_became_peered":"2024-09-17T08:32:08.529908+0000","last_unstale":"2024-09-17T08:32:38.443524+0000","last_undegraded":"2024-09-17T08:32:38.443524+0000","last_fullsized":"2024-09-17T08:32:38.443524+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:17.496846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.443933+0000","last_change":"2024-09-17T08:32:06.515825+0000","last_active":"2024-09-17T08:32:38.443933+0000","last_peered":"2024-09-17T08:32:38.443933+0000","last_clean":"2024-09-17T08:32:38.443933+0000","last_became_active":"2024-09-17T08:32:06.515261+0000","last_became_peered":"2024-09-17T08:32:06.515261+0000","last_unstale":"2024-09-17T08:32:38.443933+0000","last_undegraded":"2024-09-17T08:32:38.443933+0000","last_fullsized":"2024-09-17T08:32:38.443933+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:33:22.230484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.444216+0000","last_change":"2024-09-17T08:32:10.539278+0000","last_active":"2024-09-17T08:32:38.444216+0000","last_peered":"2024-09-17T08:32:38.444216+0000","last_clean":"2024-09-17T08:32:38.444216+0000","last_became_active":"2024-09-17T08:32:10.538963+0000","last_became_peered":"2024-09-17T08:32:10.538963+0000","last_unstale":"2024-09-17T08:32:38.444216+0000","last_undegraded":"2024-09-17T08:32:38.444216+0000","last_fullsized":"2024-09-17T08:32:38.444216+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:45:25.115543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441188+0000","last_change":"2024-09-17T08:32:12.543598+0000","last_active":"2024-09-17T08:32:38.441188+0000","last_peered":"2024-09-17T08:32:38.441188+0000","last_clean":"2024-09-17T08:32:38.441188+0000","last_became_active":"2024-09-17T08:32:12.543357+0000","last_became_peered":"2024-09-17T08:32:12.543357+0000","last_unstale":"2024-09-17T08:32:38.441188+0000","last_undegraded":"2024-09-17T08:32:38.441188+0000","last_fullsized":"2024-09-17T08:32:38.441188+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:29:40.676098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"67'9","reported_seq":35,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.444560+0000","last_change":"2024-09-17T08:32:08.528603+0000","last_active":"2024-09-17T08:32:38.444560+0000","last_peered":"2024-09-17T08:32:38.444560+0000","last_clean":"2024-09-17T08:32:38.444560+0000","last_became_active":"2024-09-17T08:32:08.528248+0000","last_became_peered":"2024-09-17T08:32:08.528248+0000","last_unstale":"2024-09-17T08:32:38.444560+0000","last_undegraded":"2024-09-17T08:32:38.444560+0000","last_fullsized":"2024-09-17T08:32:38.444560+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:47.118008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439803+0000","last_change":"2024-09-17T08:32:06.523347+0000","last_active":"2024-09-17T08:32:38.439803+0000","last_peered":"2024-09-17T08:32:38.439803+0000","last_clean":"2024-09-17T08:32:38.439803+0000","last_became_active":"2024-09-17T08:32:06.523056+0000","last_became_peered":"2024-09-17T08:32:06.523056+0000","last_unstale":"2024-09-17T08:32:38.439803+0000","last_undegraded":"2024-09-17T08:32:38.439803+0000","last_fullsized":"2024-09-17T08:32:38.439803+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:32:44.984661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.440778+0000","last_change":"2024-09-17T08:32:10.539856+0000","last_active":"2024-09-17T08:32:38.440778+0000","last_peered":"2024-09-17T08:32:38.440778+0000","last_clean":"2024-09-17T08:32:38.440778+0000","last_became_active":"2024-09-17T08:32:10.539566+0000","last_became_peered":"2024-09-17T08:32:10.539566+0000","last_unstale":"2024-09-17T08:32:38.440778+0000","last_undegraded":"2024-09-17T08:32:38.440778+0000","last_fullsized":"2024-09-17T08:32:38.440778+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:49:57.827911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"67'1","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439298+0000","last_change":"2024-09-17T08:32:12.546667+0000","last_active":"2024-09-17T08:32:38.439298+0000","last_peered":"2024-09-17T08:32:38.439298+0000","last_clean":"2024-09-17T08:32:38.439298+0000","last_became_active":"2024-09-17T08:32:12.542401+0000","last_became_peered":"2024-09-17T08:32:12.542401+0000","last_unstale":"2024-09-17T08:32:38.439298+0000","last_undegraded":"2024-09-17T08:32:38.439298+0000","last_fullsized":"2024-09-17T08:32:38.439298+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:40:18.194600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"67'11","reported_seq":38,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441352+0000","last_change":"2024-09-17T08:32:08.529822+0000","last_active":"2024-09-17T08:32:38.441352+0000","last_peered":"2024-09-17T08:32:38.441352+0000","last_clean":"2024-09-17T08:32:38.441352+0000","last_became_active":"2024-09-17T08:32:08.529291+0000","last_became_peered":"2024-09-17T08:32:08.529291+0000","last_unstale":"2024-09-17T08:32:38.441352+0000","last_undegraded":"2024-09-17T08:32:38.441352+0000","last_fullsized":"2024-09-17T08:32:38.441352+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:06:43.050801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.441833+0000","last_change":"2024-09-17T08:32:06.516834+0000","last_active":"2024-09-17T08:32:38.441833+0000","last_peered":"2024-09-17T08:32:38.441833+0000","last_clean":"2024-09-17T08:32:38.441833+0000","last_became_active":"2024-09-17T08:32:06.515253+0000","last_became_peered":"2024-09-17T08:32:06.515253+0000","last_unstale":"2024-09-17T08:32:38.441833+0000","last_undegraded":"2024-09-17T08:32:38.441833+0000","last_fullsized":"2024-09-17T08:32:38.441833+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:25.703092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439212+0000","last_change":"2024-09-17T08:32:10.542663+0000","last_active":"2024-09-17T08:32:38.439212+0000","last_peered":"2024-09-17T08:32:38.439212+0000","last_clean":"2024-09-17T08:32:38.439212+0000","last_became_active":"2024-09-17T08:32:10.542393+0000","last_became_peered":"2024-09-17T08:32:10.542393+0000","last_unstale":"2024-09-17T08:32:38.439212+0000","last_undegraded":"2024-09-17T08:32:38.439212+0000","last_fullsized":"2024-09-17T08:32:38.439212+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:12:17.412003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.440930+0000","last_change":"2024-09-17T08:32:12.694792+0000","last_active":"2024-09-17T08:32:38.440930+0000","last_peered":"2024-09-17T08:32:38.440930+0000","last_clean":"2024-09-17T08:32:38.440930+0000","last_became_active":"2024-09-17T08:32:12.694338+0000","last_became_peered":"2024-09-17T08:32:12.694338+0000","last_unstale":"2024-09-17T08:32:38.440930+0000","last_undegraded":"2024-09-17T08:32:38.440930+0000","last_fullsized":"2024-09-17T08:32:38.440930+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:38:31.715989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"67'4","reported_seq":25,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439437+0000","last_change":"2024-09-17T08:32:08.532041+0000","last_active":"2024-09-17T08:32:38.439437+0000","last_peered":"2024-09-17T08:32:38.439437+0000","last_clean":"2024-09-17T08:32:38.439437+0000","last_became_active":"2024-09-17T08:32:08.531798+0000","last_became_peered":"2024-09-17T08:32:08.531798+0000","last_unstale":"2024-09-17T08:32:38.439437+0000","last_undegraded":"2024-09-17T08:32:38.439437+0000","last_fullsized":"2024-09-17T08:32:38.439437+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:43:51.982442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.440829+0000","last_change":"2024-09-17T08:32:06.519248+0000","last_active":"2024-09-17T08:32:38.440829+0000","last_peered":"2024-09-17T08:32:38.440829+0000","last_clean":"2024-09-17T08:32:38.440829+0000","last_became_active":"2024-09-17T08:32:06.518972+0000","last_became_peered":"2024-09-17T08:32:06.518972+0000","last_unstale":"2024-09-17T08:32:38.440829+0000","last_undegraded":"2024-09-17T08:32:38.440829+0000","last_fullsized":"2024-09-17T08:32:38.440829+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:44:47.314633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.445792+0000","last_change":"2024-09-17T08:32:10.538936+0000","last_active":"2024-09-17T08:32:38.445792+0000","last_peered":"2024-09-17T08:32:38.445792+0000","last_clean":"2024-09-17T08:32:38.445792+0000","last_became_active":"2024-09-17T08:32:10.538579+0000","last_became_peered":"2024-09-17T08:32:10.538579+0000","last_unstale":"2024-09-17T08:32:38.445792+0000","last_undegraded":"2024-09-17T08:32:38.445792+0000","last_fullsized":"2024-09-17T08:32:38.445792+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:20:44.477204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439337+0000","last_change":"2024-09-17T08:32:12.549250+0000","last_active":"2024-09-17T08:32:38.439337+0000","last_peered":"2024-09-17T08:32:38.439337+0000","last_clean":"2024-09-17T08:32:38.439337+0000","last_became_active":"2024-09-17T08:32:12.548998+0000","last_became_peered":"2024-09-17T08:32:12.548998+0000","last_unstale":"2024-09-17T08:32:38.439337+0000","last_undegraded":"2024-09-17T08:32:38.439337+0000","last_fullsized":"2024-09-17T08:32:38.439337+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:43:27.021397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.444180+0000","last_change":"2024-09-17T08:32:12.544439+0000","last_active":"2024-09-17T08:32:38.444180+0000","last_peered":"2024-09-17T08:32:38.444180+0000","last_clean":"2024-09-17T08:32:38.444180+0000","last_became_active":"2024-09-17T08:32:12.544146+0000","last_became_peered":"2024-09-17T08:32:12.544146+0000","last_unstale":"2024-09-17T08:32:38.444180+0000","last_undegraded":"2024-09-17T08:32:38.444180+0000","last_fullsized":"2024-09-17T08:32:38.444180+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:11.525079+0000","last_clean_scrub_stamp":"2024-09-17T08:32:11.525079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:03:12.011989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.439658+0000","last_change":"2024-09-17T08:32:06.519681+0000","last_active":"2024-09-17T08:32:38.439658+0000","last_peered":"2024-09-17T08:32:38.439658+0000","last_clean":"2024-09-17T08:32:38.439658+0000","last_became_active":"2024-09-17T08:32:06.519097+0000","last_became_peered":"2024-09-17T08:32:06.519097+0000","last_unstale":"2024-09-17T08:32:38.439658+0000","last_undegraded":"2024-09-17T08:32:38.439658+0000","last_fullsized":"2024-09-17T08:32:38.439658+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:05.495240+0000","last_clean_scrub_stamp":"2024-09-17T08:32:05.495240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:12:14.239889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"67'11","reported_seq":38,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.446389+0000","last_change":"2024-09-17T08:32:08.529302+0000","last_active":"2024-09-17T08:32:38.446389+0000","last_peered":"2024-09-17T08:32:38.446389+0000","last_clean":"2024-09-17T08:32:38.446389+0000","last_became_active":"2024-09-17T08:32:08.528998+0000","last_became_peered":"2024-09-17T08:32:08.528998+0000","last_unstale":"2024-09-17T08:32:38.446389+0000","last_undegraded":"2024-09-17T08:32:38.446389+0000","last_fullsized":"2024-09-17T08:32:38.446389+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:07.504893+0000","last_clean_scrub_stamp":"2024-09-17T08:32:07.504893+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:51:00.009253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T08:32:38.440937+0000","last_change":"2024-09-17T08:32:10.543482+0000","last_active":"2024-09-17T08:32:38.440937+0000","last_peered":"2024-09-17T08:32:38.440937+0000","last_clean":"2024-09-17T08:32:38.440937+0000","last_became_active":"2024-09-17T08:32:10.543158+0000","last_became_peered":"2024-09-17T08:32:10.543158+0000","last_unstale":"2024-09-17T08:32:38.440937+0000","last_undegraded":"2024-09-17T08:32:38.440937+0000","last_fullsized":"2024-09-17T08:32:38.440937+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:09.514931+0000","last_clean_scrub_stamp":"2024-09-17T08:32:09.514931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:20:53.093873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":392,"ondisk_log_size":392,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":77,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234007,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28376,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716776,"statfs":{"total":95995035648,"available":95965978624,"internally_reserved":0,"allocated":1392640,"data_stored":923806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.66500000000000004}]},{"osd":1,"last update":"Tue Sep 17 08:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.49199999999999999}]},{"osd":2,"last update":"Tue Sep 17 08:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.68000000000000005}]},{"osd":3,"last update":"Tue Sep 17 08:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.72799999999999998}]},{"osd":4,"last update":"Tue Sep 17 08:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.41299999999999998}]},{"osd":5,"last update":"Tue Sep 17 08:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.36399999999999999}]},{"osd":6,"last update":"Tue Sep 17 08:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.44500000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495644,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28344,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716808,"statfs":{"total":95995035648,"available":95966011392,"internally_reserved":0,"allocated":1359872,"data_stored":921424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:33:31 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53500000000000003,"15min":0.53400000000000003},"min":{"1min":0.39600000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76800000000000002,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54300000000000004,"15min":0.53800000000000003},"min":{"1min":0.42399999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.73599999999999999,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.35199999999999998}]},{"osd":1,"last update":"Tue Sep 17 08:33:31 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.378,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72199999999999998,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54300000000000004,"15min":0.53800000000000003},"min":{"1min":0.42499999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.70299999999999996,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.48299999999999998}]},{"osd":2,"last update":"Tue Sep 17 08:33:31 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.41099999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.79200000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52800000000000002,"15min":0.52700000000000002},"min":{"1min":0.38200000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.751,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.33100000000000002}]},{"osd":3,"last update":"Tue Sep 17 08:33:31 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.40400000000000003,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.71499999999999997,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.55200000000000005,"15min":0.55300000000000005},"min":{"1min":0.39500000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.749,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.36899999999999999}]},{"osd":4,"last update":"Tue Sep 17 08:33:31 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.42699999999999999,"15min":0.42199999999999999},"min":{"1min":0.29099999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.42599999999999999,"15min":0.41999999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.57999999999999996}]},{"osd":5,"last update":"Tue Sep 17 08:33:31 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.432,"15min":0.42599999999999999},"min":{"1min":0.29499999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.63100000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.501},{"interface":"front","average":{"1min":0.5,"5min":0.46400000000000002,"15min":0.45800000000000002},"min":{"1min":0.32400000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.433}]},{"osd":7,"last update":"Tue Sep 17 08:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.55500000000000005}]}]},{"osd":5,"up_from":38,"seq":163208757281,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27800,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717352,"statfs":{"total":95995035648,"available":95966568448,"internally_reserved":0,"allocated":798720,"data_stored":331988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:33:12 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.55400000000000005,"15min":0.56000000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.71599999999999997,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.503},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.53800000000000003,"15min":0.54600000000000004},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.75700000000000001,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.58399999999999996}]},{"osd":1,"last update":"Tue Sep 17 08:33:12 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.55100000000000005,"15min":0.55800000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77500000000000002,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.56799999999999995,"15min":0.57699999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.69999999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.55700000000000005}]},{"osd":2,"last update":"Tue Sep 17 08:33:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57799999999999996,"15min":0.58299999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.74399999999999999,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.56000000000000005,"15min":0.56699999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.72999999999999998,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.52800000000000002}]},{"osd":3,"last update":"Tue Sep 17 08:33:12 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.54500000000000004,"15min":0.55200000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68700000000000006,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56999999999999995,"15min":0.57399999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.72099999999999997,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.54100000000000004}]},{"osd":4,"last update":"Tue Sep 17 08:33:12 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.41899999999999998,"15min":0.41699999999999998},"min":{"1min":0.28299999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.64900000000000002,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.30599999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.628,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.71499999999999997}]},{"osd":6,"last update":"Tue Sep 17 08:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.45100000000000001,"15min":0.44800000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.69499999999999995,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.46100000000000002,"15min":0.45600000000000002},"min":{"1min":0.308,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.75,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.63600000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:32:54 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.62}]}]},{"osd":4,"up_from":30,"seq":128849018918,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31892,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713260,"statfs":{"total":95995035648,"available":95962378240,"internally_reserved":0,"allocated":794624,"data_stored":332026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.54100000000000004,"15min":0.54900000000000004},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.51900000000000002,"15min":0.52100000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70599999999999996,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.42999999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52700000000000002,"15min":0.52400000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54700000000000004,"15min":0.54300000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.58699999999999997}]},{"osd":2,"last update":"Tue Sep 17 08:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.54800000000000004,"15min":0.55000000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71199999999999997,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.52000000000000002,"15min":0.52500000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.64500000000000002,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.54200000000000004}]},{"osd":3,"last update":"Tue Sep 17 08:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53200000000000003,"15min":0.52900000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.502},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.54300000000000004},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.41099999999999998}]},{"osd":5,"last update":"Tue Sep 17 08:33:13 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.38900000000000001,"15min":0.379},"min":{"1min":0.26500000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.40899999999999997,"15min":0.39700000000000002},"min":{"1min":0.32100000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.47599999999999998}]},{"osd":6,"last update":"Tue Sep 17 08:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.38800000000000001,"15min":0.376},"min":{"1min":0.27800000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.46700000000000003,"15min":0.45800000000000002},"min":{"1min":0.38200000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.55500000000000005}]},{"osd":7,"last update":"Tue Sep 17 08:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.374},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.48899999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182441,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31908,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713244,"statfs":{"total":95995035648,"available":95962361856,"internally_reserved":0,"allocated":811008,"data_stored":333232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:33:26 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.377,"15min":0.38900000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.375,"5min":0.39100000000000001,"15min":0.40000000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.29199999999999998}]},{"osd":1,"last update":"Tue Sep 17 08:33:26 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.40000000000000002,"15min":0.40600000000000003},"min":{"1min":0.254,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.39800000000000002,"15min":0.40899999999999997},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.36699999999999999}]},{"osd":2,"last update":"Tue Sep 17 08:33:26 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.42999999999999999,"15min":0.44500000000000001},"min":{"1min":0.29999999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.53900000000000003,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.379,"5min":0.39500000000000002,"15min":0.40400000000000003},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.27400000000000002}]},{"osd":4,"last update":"Tue Sep 17 08:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.47399999999999998}]},{"osd":5,"last update":"Tue Sep 17 08:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.40500000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.443}]},{"osd":6,"last update":"Tue Sep 17 08:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56899999999999995,"15min":0.57099999999999995},"min":{"1min":0.40300000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.46000000000000002}]},{"osd":7,"last update":"Tue Sep 17 08:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.55000000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411375,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27780,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717372,"statfs":{"total":95995035648,"available":95966588928,"internally_reserved":0,"allocated":778240,"data_stored":332157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:32:58 2024","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.33100000000000002,"15min":0.32000000000000001},"min":{"1min":0.20899999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.41499999999999998,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.35299999999999998,"15min":0.34200000000000003},"min":{"1min":0.223,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.49199999999999999,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.26700000000000002}]},{"osd":1,"last update":"Tue Sep 17 08:32:58 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.34599999999999997,"15min":0.33600000000000002},"min":{"1min":0.23799999999999999,"5min":0.216,"15min":0.216},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.36499999999999999,"15min":0.35699999999999998},"min":{"1min":0.24399999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.45300000000000001,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.33200000000000002}]},{"osd":3,"last update":"Tue Sep 17 08:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.39900000000000002,"15min":0.40699999999999997},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.41099999999999998,"15min":0.41899999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.54200000000000004,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.29999999999999999}]},{"osd":4,"last update":"Tue Sep 17 08:32:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56200000000000006,"15min":0.55600000000000005},"min":{"1min":0.42299999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54600000000000004,"15min":0.54400000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.71699999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.436}]},{"osd":5,"last update":"Tue Sep 17 08:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57199999999999995,"15min":0.56999999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.317},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.41999999999999998}]},{"osd":6,"last update":"Tue Sep 17 08:33:31 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.627,"15min":0.64200000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.75900000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77200000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.35899999999999999}]},{"osd":7,"last update":"Tue Sep 17 08:32:55 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.374},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.39800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":815104,"data_stored":333662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.32700000000000001},"min":{"1min":0.20000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.44500000000000001,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.35599999999999998,"15min":0.33500000000000002},"min":{"1min":0.23799999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.53500000000000003,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.505}]},{"osd":2,"last update":"Tue Sep 17 08:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.38100000000000001,"15min":0.375},"min":{"1min":0.20499999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.50600000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.42399999999999999,"15min":0.41999999999999998},"min":{"1min":0.28499999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.60199999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:33:25 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.39800000000000002,"15min":0.40600000000000003},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.48399999999999999,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.42999999999999999,"15min":0.435},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.53900000000000003,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.42899999999999999}]},{"osd":4,"last update":"Tue Sep 17 08:32:45 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54200000000000004,"15min":0.54000000000000004},"min":{"1min":0.35299999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53600000000000003,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.68200000000000005,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.41799999999999998}]},{"osd":5,"last update":"Tue Sep 17 08:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.60299999999999998,"15min":0.61399999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.59099999999999997,"15min":0.60599999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.65800000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.38500000000000001}]},{"osd":6,"last update":"Tue Sep 17 08:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.56799999999999995,"15min":0.57599999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.68400000000000005,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.59499999999999997,"15min":0.60299999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.51800000000000002}]},{"osd":7,"last update":"Tue Sep 17 08:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.44800000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738425,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28380,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716772,"statfs":{"total":95995035648,"available":95965974528,"internally_reserved":0,"allocated":1392640,"data_stored":922642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 17 08:32:35 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.33300000000000002,"15min":0.313},"min":{"1min":0.24399999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.52400000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.33900000000000002,"15min":0.314},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.249}]},{"osd":2,"last update":"Tue Sep 17 08:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.40000000000000002,"15min":0.39400000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.50800000000000001,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.40000000000000002,"15min":0.39800000000000002},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.53200000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.32400000000000001}]},{"osd":3,"last update":"Tue Sep 17 08:33:26 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.44400000000000001,"15min":0.45600000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.61799999999999999,"5min":0.629,"15min":0.629},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.46300000000000002,"15min":0.48699999999999999},"min":{"1min":0.28299999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.621,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.34699999999999998}]},{"osd":4,"last update":"Tue Sep 17 08:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.39400000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.73999999999999999,"5min":0.754,"15min":0.754},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57299999999999995,"15min":0.57199999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.51600000000000001}]},{"osd":5,"last update":"Tue Sep 17 08:33:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57799999999999996,"15min":0.57699999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.41099999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.73299999999999998,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.442}]},{"osd":6,"last update":"Tue Sep 17 08:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57599999999999996,"15min":0.56899999999999995},"min":{"1min":0.44,"5min":0.377,"15min":0.377},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.45100000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.40000000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T08:33:38.435 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-17T08:33:38.435 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-17T08:33:38.435 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-17T08:33:38.436 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-17T08:33:38.530 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:33:38 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:33:38.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:38 smithi118 ceph-mon[29491]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:38.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:38 smithi118 ceph-mon[41418]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:38.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:38 smithi170 ceph-mon[33271]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:39.071 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:40.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:40.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:40.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:40.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:40.278 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 podman[77448]: 2024-09-17 08:33:40.164232762 +0000 UTC m=+3.157268891 container create d20995b52b5afacca5db23658cba21ddb8e394fc3ed6ec3c1f1619e3465d3b65 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-17T08:33:40.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:40 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:40.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:40 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:41.110 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 podman[77448]: 2024-09-17 08:33:40.953383809 +0000 UTC m=+3.946419936 container init d20995b52b5afacca5db23658cba21ddb8e394fc3ed6ec3c1f1619e3465d3b65 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-17T08:33:41.110 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 podman[77448]: 2024-09-17 08:33:40.958949441 +0000 UTC m=+3.951985568 container start d20995b52b5afacca5db23658cba21ddb8e394fc3ed6ec3c1f1619e3465d3b65 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-17T08:33:41.110 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.962Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-17T08:33:41.110 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.962Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-17T08:33:41.110 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.963Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-17T08:33:41.110 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.963Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-17T08:33:41.110 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.963Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-17T08:33:41.110 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.963Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-17T08:33:41.110 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-17T08:33:41.110 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=arp 2024-09-17T08:33:41.110 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-17T08:33:41.111 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-17T08:33:41.111 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-17T08:33:41.111 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-17T08:33:41.111 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-17T08:33:41.111 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-17T08:33:41.111 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-17T08:33:41.112 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-17T08:33:41.112 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=edac 2024-09-17T08:33:41.112 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-17T08:33:41.112 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-17T08:33:41.112 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-17T08:33:41.112 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-17T08:33:41.112 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-17T08:33:41.112 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-17T08:33:41.112 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-17T08:33:41.112 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-17T08:33:41.112 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-17T08:33:41.112 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-17T08:33:41.112 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-17T08:33:41.112 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-17T08:33:41.113 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-17T08:33:41.113 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-17T08:33:41.113 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-17T08:33:41.113 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-17T08:33:41.113 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=os 2024-09-17T08:33:41.113 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-17T08:33:41.113 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-17T08:33:41.113 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-17T08:33:41.113 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-17T08:33:41.113 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-17T08:33:41.113 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-17T08:33:41.113 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-17T08:33:41.113 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=stat 2024-09-17T08:33:41.113 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-17T08:33:41.114 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-17T08:33:41.114 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-17T08:33:41.114 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=time 2024-09-17T08:33:41.114 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-17T08:33:41.114 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=uname 2024-09-17T08:33:41.114 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-17T08:33:41.114 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-17T08:33:41.114 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-17T08:33:41.114 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-17T08:33:41.114 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:40 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a[77661]: ts=2024-09-17T08:33:40.964Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-17T08:33:41.114 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:41 smithi118 bash[77448]: d20995b52b5afacca5db23658cba21ddb8e394fc3ed6ec3c1f1619e3465d3b65 2024-09-17T08:33:41.114 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:33:41 smithi118 systemd[1]: Started Ceph node-exporter.a for 7862d192-74ce-11ef-bceb-c7b262605968. 2024-09-17T08:33:41.365 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:41 smithi118 ceph-mon[29491]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:41.366 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:41 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:41.366 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:41 smithi118 ceph-mon[41418]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:41.366 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:41 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:41.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:41 smithi170 ceph-mon[33271]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:41.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:41 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:41.759 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-17T08:33:41.760 INFO:teuthology.orchestra.run.smithi118.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-17T08:33:42.116 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:42 smithi170 bash[66178]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-17T08:33:42.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:42 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:42.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:42 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:42.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:42 smithi170 ceph-mon[33271]: Deploying daemon node-exporter.b on smithi170 2024-09-17T08:33:42.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:42 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/1940874734' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-17T08:33:42.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:42 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:42.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:42 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:42.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:42 smithi118 ceph-mon[29491]: Deploying daemon node-exporter.b on smithi170 2024-09-17T08:33:42.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:42 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/1940874734' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-17T08:33:42.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:42 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:42.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:42 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:42.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:42 smithi118 ceph-mon[41418]: Deploying daemon node-exporter.b on smithi170 2024-09-17T08:33:42.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:42 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/1940874734' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-17T08:33:42.895 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:42 smithi170 bash[66178]: Getting image source signatures 2024-09-17T08:33:42.895 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:42 smithi170 bash[66178]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-17T08:33:42.895 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:42 smithi170 bash[66178]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-17T08:33:42.895 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:42 smithi170 bash[66178]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-17T08:33:43.024 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-17T08:33:43.024 INFO:tasks.cephadm:Setup complete, yielding 2024-09-17T08:33:43.025 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-17T08:33:43.034 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi118.front.sepia.ceph.com 2024-09-17T08:33:43.035 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7862d192-74ce-11ef-bceb-c7b262605968 -- bash -c 'set -ex 2024-09-17T08:33:43.035 DEBUG:teuthology.orchestra.run.smithi118:> for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-17T08:33:43.035 DEBUG:teuthology.orchestra.run.smithi118:> do 2024-09-17T08:33:43.036 DEBUG:teuthology.orchestra.run.smithi118:> echo "rotating key for $f" 2024-09-17T08:33:43.036 DEBUG:teuthology.orchestra.run.smithi118:> K=$(ceph auth get-key $f) 2024-09-17T08:33:43.036 DEBUG:teuthology.orchestra.run.smithi118:> NK="$K" 2024-09-17T08:33:43.036 DEBUG:teuthology.orchestra.run.smithi118:> ceph orch daemon rotate-key $f 2024-09-17T08:33:43.036 DEBUG:teuthology.orchestra.run.smithi118:> while [ "$K" == "$NK" ]; do 2024-09-17T08:33:43.036 DEBUG:teuthology.orchestra.run.smithi118:> sleep 5 2024-09-17T08:33:43.036 DEBUG:teuthology.orchestra.run.smithi118:> NK=$(ceph auth get-key $f) 2024-09-17T08:33:43.036 DEBUG:teuthology.orchestra.run.smithi118:> done 2024-09-17T08:33:43.036 DEBUG:teuthology.orchestra.run.smithi118:> done 2024-09-17T08:33:43.036 DEBUG:teuthology.orchestra.run.smithi118:> ' 2024-09-17T08:33:43.245 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:43.347 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:43 smithi118 ceph-mon[29491]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:43.347 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:43 smithi118 ceph-mon[41418]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:43.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:43 smithi170 ceph-mon[33271]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:44.395 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 bash[66178]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-17T08:33:44.395 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 bash[66178]: Writing manifest to image destination 2024-09-17T08:33:44.840 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 podman[66178]: 2024-09-17 08:33:44.497420435 +0000 UTC m=+2.569271562 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-17T08:33:44.840 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 podman[66178]: 2024-09-17 08:33:44.62108502 +0000 UTC m=+2.692936137 container create c1c963b1d7f20ed903ba446157960befc60f047d3ed90b59bf054a2847cba533 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-17T08:33:44.840 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 podman[66178]: 2024-09-17 08:33:44.840429483 +0000 UTC m=+2.912280600 container init c1c963b1d7f20ed903ba446157960befc60f047d3ed90b59bf054a2847cba533 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-17T08:33:45.122 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 podman[66178]: 2024-09-17 08:33:44.845233358 +0000 UTC m=+2.917084478 container start c1c963b1d7f20ed903ba446157960befc60f047d3ed90b59bf054a2847cba533 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-17T08:33:45.122 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.849Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-17T08:33:45.123 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.849Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-17T08:33:45.123 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.849Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-17T08:33:45.123 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-17T08:33:45.123 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-17T08:33:45.123 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-17T08:33:45.123 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-17T08:33:45.123 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=arp 2024-09-17T08:33:45.123 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-17T08:33:45.123 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-17T08:33:45.123 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-17T08:33:45.123 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-17T08:33:45.123 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-17T08:33:45.124 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-17T08:33:45.124 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-17T08:33:45.124 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-17T08:33:45.124 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=edac 2024-09-17T08:33:45.124 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-17T08:33:45.124 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-17T08:33:45.124 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-17T08:33:45.124 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-17T08:33:45.124 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-17T08:33:45.124 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-17T08:33:45.124 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-17T08:33:45.124 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-17T08:33:45.124 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-17T08:33:45.125 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-17T08:33:45.125 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-17T08:33:45.125 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-17T08:33:45.125 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-17T08:33:45.125 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-17T08:33:45.126 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-17T08:33:45.126 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-17T08:33:45.126 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=os 2024-09-17T08:33:45.126 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-17T08:33:45.126 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-17T08:33:45.127 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-17T08:33:45.127 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-17T08:33:45.127 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-17T08:33:45.127 INFO:teuthology.orchestra.run.smithi118.stdout:rotating key for osd.0 2024-09-17T08:33:45.128 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-17T08:33:45.128 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-17T08:33:45.128 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=stat 2024-09-17T08:33:45.128 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-17T08:33:45.128 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-17T08:33:45.128 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-17T08:33:45.128 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=time 2024-09-17T08:33:45.128 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-17T08:33:45.128 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=uname 2024-09-17T08:33:45.128 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-17T08:33:45.129 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-17T08:33:45.129 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-17T08:33:45.129 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-17T08:33:45.129 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b[66252]: ts=2024-09-17T08:33:44.850Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-17T08:33:45.129 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 bash[66178]: c1c963b1d7f20ed903ba446157960befc60f047d3ed90b59bf054a2847cba533 2024-09-17T08:33:45.129 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:33:44 smithi170 systemd[1]: Started Ceph node-exporter.b for 7862d192-74ce-11ef-bceb-c7b262605968. 2024-09-17T08:33:45.129 INFO:teuthology.orchestra.run.smithi118.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-17T08:33:45.129 INFO:teuthology.orchestra.run.smithi118.stderr:+ echo 'rotating key for osd.0' 2024-09-17T08:33:45.129 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:33:45.360 INFO:teuthology.orchestra.run.smithi118.stderr:+ K=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:33:45.361 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:33:45.361 INFO:teuthology.orchestra.run.smithi118.stderr:+ ceph orch daemon rotate-key osd.0 2024-09-17T08:33:45.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:45 smithi170 ceph-mon[33271]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:45.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:45 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:33:45.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:45 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:45.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:45 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:45.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:45 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:45.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:45 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:45.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:45 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:45.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:45 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:33:45.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:45 smithi118 ceph-mon[41418]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:45.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:45 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:33:45.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:45 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:45.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:45 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:45.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:45 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:45.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:45 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:45.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:45 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:45.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:45 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:33:45.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:45 smithi118 ceph-mon[29491]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:45.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:45 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:33:45.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:45 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:45.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:45 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:45.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:45 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:45.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:45 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:45.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:45 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:45.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:45 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:33:45.633 INFO:teuthology.orchestra.run.smithi118.stdout:Scheduled to rotate-key osd.0 on host 'smithi118' 2024-09-17T08:33:45.646 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:33:45.646 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:33:46.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:46 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/466930798' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:33:46.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:46 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:46.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:46 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:46.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:46 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/466930798' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:33:46.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:46 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:46.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:46 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:46.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:46 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/466930798' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:33:46.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:46 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:46.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:46 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:47.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:47 smithi170 ceph-mon[33271]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:33:47.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:47 smithi170 ceph-mon[33271]: Schedule rotate-key daemon osd.0 2024-09-17T08:33:47.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:47 smithi170 ceph-mon[33271]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:47.527 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:33:47 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:33:47] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:33:47.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:47 smithi118 ceph-mon[29491]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:33:47.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:47 smithi118 ceph-mon[29491]: Schedule rotate-key daemon osd.0 2024-09-17T08:33:47.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:47 smithi118 ceph-mon[29491]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:47.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:47 smithi118 ceph-mon[41418]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:33:47.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:47 smithi118 ceph-mon[41418]: Schedule rotate-key daemon osd.0 2024-09-17T08:33:47.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:47 smithi118 ceph-mon[41418]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:48.645 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:33:48 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:33:49.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:49 smithi170 ceph-mon[33271]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:49.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:49 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:49.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:49 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:49.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:49 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:49.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:49 smithi118 ceph-mon[29491]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:49.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:49 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:49.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:49 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:49.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:49 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:49.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:49 smithi118 ceph-mon[41418]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:49.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:49 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:49.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:49 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:49.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:49 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:50.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:50 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:50.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:50 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:50.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:50 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:50.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:50 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:33:50.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:50 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:33:50.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:50 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:50.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:50 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]: dispatch 2024-09-17T08:33:50.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:50 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]: dispatch 2024-09-17T08:33:50.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:50 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]': finished 2024-09-17T08:33:50.353 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:50 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:50.353 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:50 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:50.353 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:50 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:50.353 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:50 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:33:50.354 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:50 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:33:50.354 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:50 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:50.354 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:50 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]: dispatch 2024-09-17T08:33:50.354 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:50 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]: dispatch 2024-09-17T08:33:50.354 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:50 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]': finished 2024-09-17T08:33:50.354 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:50 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:50.354 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:50 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:50.354 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:50 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:50.354 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:50 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:33:50.354 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:50 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:33:50.354 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:50 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:50.354 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:50 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]: dispatch 2024-09-17T08:33:50.355 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:50 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]: dispatch 2024-09-17T08:33:50.355 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:50 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]': finished 2024-09-17T08:33:50.648 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:33:50.966 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:33:50.966 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:33:50.966 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:33:51.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:51 smithi118 ceph-mon[29491]: Rotating authentication key for osd.0 2024-09-17T08:33:51.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:51 smithi118 ceph-mon[29491]: Reconfiguring daemon osd.0 on smithi118 2024-09-17T08:33:51.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:51 smithi118 ceph-mon[29491]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:51.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:51 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/1125934459' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:33:51.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:51 smithi118 ceph-mon[41418]: Rotating authentication key for osd.0 2024-09-17T08:33:51.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:51 smithi118 ceph-mon[41418]: Reconfiguring daemon osd.0 on smithi118 2024-09-17T08:33:51.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:51 smithi118 ceph-mon[41418]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:51.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:51 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/1125934459' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:33:51.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:51 smithi170 ceph-mon[33271]: Rotating authentication key for osd.0 2024-09-17T08:33:51.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:51 smithi170 ceph-mon[33271]: Reconfiguring daemon osd.0 on smithi118 2024-09-17T08:33:51.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:51 smithi170 ceph-mon[33271]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:51.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:51 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/1125934459' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:33:53.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:52 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:53.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:52 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:53.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:52 smithi118 ceph-mon[29491]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:53.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:52 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:53.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:52 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:53.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:52 smithi118 ceph-mon[29491]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-17T08:33:53.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:52 smithi118 ceph-mon[29491]: Reconfiguring daemon alertmanager.a on smithi118 2024-09-17T08:33:53.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:52 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:53.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:52 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:53.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:52 smithi118 ceph-mon[41418]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:53.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:52 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:53.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:52 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:53.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:52 smithi118 ceph-mon[41418]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-17T08:33:53.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:52 smithi118 ceph-mon[41418]: Reconfiguring daemon alertmanager.a on smithi118 2024-09-17T08:33:53.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:52 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:53.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:52 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:53.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:52 smithi170 ceph-mon[33271]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:53.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:52 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:53.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:52 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:53.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:52 smithi170 ceph-mon[33271]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-17T08:33:53.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:52 smithi170 ceph-mon[33271]: Reconfiguring daemon alertmanager.a on smithi118 2024-09-17T08:33:55.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:55 smithi170 ceph-mon[33271]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:55.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:55 smithi118 ceph-mon[29491]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:55.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:55 smithi118 ceph-mon[41418]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:55.528 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:55 smithi118 systemd[1]: Stopping Ceph alertmanager.a for 7862d192-74ce-11ef-bceb-c7b262605968... 2024-09-17T08:33:55.881 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:55 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a[76364]: ts=2024-09-17T08:33:55.532Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-17T08:33:55.882 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:55 smithi118 podman[79099]: 2024-09-17 08:33:55.563688566 +0000 UTC m=+0.135605767 container died fa53fcaa2dfa341d54eb14bd967b307405a6f6241d0ce3b001c4d268880431ce (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T08:33:55.968 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:33:56.277 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:56 smithi118 podman[79099]: 2024-09-17 08:33:56.081182489 +0000 UTC m=+0.653099681 container cleanup fa53fcaa2dfa341d54eb14bd967b307405a6f6241d0ce3b001c4d268880431ce (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T08:33:56.277 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:56 smithi118 bash[79099]: ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a 2024-09-17T08:33:56.294 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:33:56.294 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:33:56.294 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:33:56.631 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:56 smithi118 podman[79117]: 2024-09-17 08:33:56.299730422 +0000 UTC m=+0.738836315 container remove fa53fcaa2dfa341d54eb14bd967b307405a6f6241d0ce3b001c4d268880431ce (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T08:33:56.631 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:56 smithi118 podman[79117]: 2024-09-17 08:33:56.375209624 +0000 UTC m=+0.814315469 volume remove 5e09c1896cec06655e2ccb6726454e904150453407d43eeb9e6147d1b5bc6f27 2024-09-17T08:33:56.943 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:56 smithi118 systemd[1]: ceph-7862d192-74ce-11ef-bceb-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-17T08:33:56.943 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:56 smithi118 systemd[1]: Stopped Ceph alertmanager.a for 7862d192-74ce-11ef-bceb-c7b262605968. 2024-09-17T08:33:56.943 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:56 smithi118 systemd[1]: Starting Ceph alertmanager.a for 7862d192-74ce-11ef-bceb-c7b262605968... 2024-09-17T08:33:56.943 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:56 smithi118 podman[79238]: 2024-09-17 08:33:56.851310108 +0000 UTC m=+0.108005618 volume create 19a09ff9a6638b823c47a9f3ec1d090c6f119c684f680acc9d4c4fa6ec6e9d81 2024-09-17T08:33:56.943 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:56 smithi118 podman[79238]: 2024-09-17 08:33:56.759122266 +0000 UTC m=+0.015817795 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-09-17T08:33:56.943 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:56 smithi118 podman[79238]: 2024-09-17 08:33:56.943365827 +0000 UTC m=+0.200061335 container create 8b804409f7af67cbd06b25eb49686ff2a98463d016f2d60cfe3c31337326c925 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T08:33:57.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:57 smithi118 ceph-mon[29491]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:57.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:57 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/1473255777' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:33:57.277 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:33:57 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:33:57] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:33:57.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:57 smithi118 ceph-mon[41418]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:57.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:57 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/1473255777' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:33:57.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:57 smithi170 ceph-mon[33271]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:57.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:57 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/1473255777' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:33:57.777 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:57 smithi118 podman[79238]: 2024-09-17 08:33:57.329944866 +0000 UTC m=+0.586640384 container init 8b804409f7af67cbd06b25eb49686ff2a98463d016f2d60cfe3c31337326c925 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T08:33:57.778 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:57 smithi118 podman[79238]: 2024-09-17 08:33:57.334730564 +0000 UTC m=+0.591426076 container start 8b804409f7af67cbd06b25eb49686ff2a98463d016f2d60cfe3c31337326c925 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T08:33:57.778 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:57 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a[79253]: ts=2024-09-17T08:33:57.346Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-17T08:33:57.778 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:57 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a[79253]: ts=2024-09-17T08:33:57.346Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-17T08:33:57.778 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:57 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a[79253]: ts=2024-09-17T08:33:57.347Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.118 port=9094 2024-09-17T08:33:57.778 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:57 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a[79253]: ts=2024-09-17T08:33:57.348Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-17T08:33:57.778 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:57 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a[79253]: ts=2024-09-17T08:33:57.371Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-17T08:33:57.778 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:57 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a[79253]: ts=2024-09-17T08:33:57.372Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-17T08:33:57.778 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:57 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a[79253]: ts=2024-09-17T08:33:57.374Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-17T08:33:57.778 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:57 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a[79253]: ts=2024-09-17T08:33:57.374Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-17T08:33:57.778 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:57 smithi118 bash[79238]: 8b804409f7af67cbd06b25eb49686ff2a98463d016f2d60cfe3c31337326c925 2024-09-17T08:33:57.778 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:57 smithi118 systemd[1]: Started Ceph alertmanager.a for 7862d192-74ce-11ef-bceb-c7b262605968. 2024-09-17T08:33:58.314 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:33:58 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:33:58.565 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:58 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:58.565 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:58 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:58.565 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:58 smithi170 ceph-mon[33271]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-17T08:33:58.566 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:58 smithi170 ceph-mon[33271]: Reconfiguring daemon prometheus.a on smithi170 2024-09-17T08:33:58.566 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:58 smithi170 ceph-mon[33271]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:58.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:58 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:58.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:58 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:58.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:58 smithi118 ceph-mon[29491]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-17T08:33:58.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:58 smithi118 ceph-mon[29491]: Reconfiguring daemon prometheus.a on smithi170 2024-09-17T08:33:58.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:58 smithi118 ceph-mon[29491]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:58.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:58 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:58.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:58 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:33:58.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:58 smithi118 ceph-mon[41418]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-17T08:33:58.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:58 smithi118 ceph-mon[41418]: Reconfiguring daemon prometheus.a on smithi170 2024-09-17T08:33:58.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:58 smithi118 ceph-mon[41418]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:59.596 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:59 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:59.596 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:33:59 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:33:59.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:59 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:59.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:33:59 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:33:59.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:59 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:59.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:33:59 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:33:59.778 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:33:59 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a[79253]: ts=2024-09-17T08:33:59.349Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.001045413s 2024-09-17T08:34:00.703 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:00 smithi170 ceph-mon[33271]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:00.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:00 smithi118 ceph-mon[29491]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:00.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:00 smithi118 ceph-mon[41418]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:01.145 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:00 smithi170 systemd[1]: Stopping Ceph prometheus.a for 7862d192-74ce-11ef-bceb-c7b262605968... 2024-09-17T08:34:01.296 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:34:01.450 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:01 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[63929]: ts=2024-09-17T08:34:01.159Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-17T08:34:01.450 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:01 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[63929]: ts=2024-09-17T08:34:01.159Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-17T08:34:01.450 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:01 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[63929]: ts=2024-09-17T08:34:01.159Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-17T08:34:01.450 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:01 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[63929]: ts=2024-09-17T08:34:01.159Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-17T08:34:01.450 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:01 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[63929]: ts=2024-09-17T08:34:01.159Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-17T08:34:01.450 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:01 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[63929]: ts=2024-09-17T08:34:01.159Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-17T08:34:01.451 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:01 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[63929]: ts=2024-09-17T08:34:01.159Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-17T08:34:01.451 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:01 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[63929]: ts=2024-09-17T08:34:01.159Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-17T08:34:01.451 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:01 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[63929]: ts=2024-09-17T08:34:01.159Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-17T08:34:01.451 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:01 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[63929]: ts=2024-09-17T08:34:01.159Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-17T08:34:01.451 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:01 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[63929]: ts=2024-09-17T08:34:01.159Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-17T08:34:01.451 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:01 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[63929]: ts=2024-09-17T08:34:01.159Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-17T08:34:01.451 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:01 smithi170 podman[67290]: 2024-09-17 08:34:01.16995484 +0000 UTC m=+0.117564336 container died 2b28ec08861ac5156a4366c4b4921a2fbf4bc4819d11e4c130e286fc6499a18f (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T08:34:01.618 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:34:01.618 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:34:01.618 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:34:01.824 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:01 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/4172407952' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:34:01.825 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:01 smithi170 podman[67290]: 2024-09-17 08:34:01.54933217 +0000 UTC m=+0.496941664 container cleanup 2b28ec08861ac5156a4366c4b4921a2fbf4bc4819d11e4c130e286fc6499a18f (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T08:34:01.825 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:01 smithi170 bash[67290]: ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a 2024-09-17T08:34:02.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:01 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/4172407952' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:34:02.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:01 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/4172407952' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:34:02.145 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:01 smithi170 podman[67308]: 2024-09-17 08:34:01.825267044 +0000 UTC m=+0.652476858 container remove 2b28ec08861ac5156a4366c4b4921a2fbf4bc4819d11e4c130e286fc6499a18f (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T08:34:02.145 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:02 smithi170 systemd[1]: ceph-7862d192-74ce-11ef-bceb-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-09-17T08:34:02.145 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:02 smithi170 systemd[1]: Stopped Ceph prometheus.a for 7862d192-74ce-11ef-bceb-c7b262605968. 2024-09-17T08:34:02.145 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:02 smithi170 systemd[1]: ceph-7862d192-74ce-11ef-bceb-c7b262605968@prometheus.a.service: Consumed 1.185s CPU time. 2024-09-17T08:34:02.145 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:02 smithi170 systemd[1]: Starting Ceph prometheus.a for 7862d192-74ce-11ef-bceb-c7b262605968... 2024-09-17T08:34:02.473 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:02 smithi170 podman[67404]: 2024-09-17 08:34:02.113571202 +0000 UTC m=+0.016744017 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-17T08:34:02.474 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:02 smithi170 podman[67404]: 2024-09-17 08:34:02.217864419 +0000 UTC m=+0.121037221 container create 993a3bf4ddb66824af98c86898194a7f16b2386a8d7937a9a363f1aa6fefb0bb (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T08:34:02.474 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:02 smithi170 podman[67404]: 2024-09-17 08:34:02.419781839 +0000 UTC m=+0.322954641 container init 993a3bf4ddb66824af98c86898194a7f16b2386a8d7937a9a363f1aa6fefb0bb (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T08:34:02.474 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:02 smithi170 podman[67404]: 2024-09-17 08:34:02.424348645 +0000 UTC m=+0.327521460 container start 993a3bf4ddb66824af98c86898194a7f16b2386a8d7937a9a363f1aa6fefb0bb (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T08:34:02.474 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:02 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:34:02.448Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-17T08:34:02.474 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:02 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:34:02.448Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-17T08:34:02.474 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:02 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:34:02.448Z caller=main.go:570 level=info host_details="(Linux 5.14.0-508.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 12 15:49:37 UTC 2024 x86_64 smithi170 (none))" 2024-09-17T08:34:02.474 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:02 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:34:02.448Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-17T08:34:02.474 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:02 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:34:02.448Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-17T08:34:02.474 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:02 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:34:02.450Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-17T08:34:02.474 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:02 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:34:02.450Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-17T08:34:02.474 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:02 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:34:02.451Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-17T08:34:02.474 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:02 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:34:02.451Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-17T08:34:02.474 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:02 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:34:02.452Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-17T08:34:02.475 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:02 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:34:02.452Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.563µs 2024-09-17T08:34:02.475 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:02 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:34:02.452Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-17T08:34:02.475 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:02 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:34:02.453Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-09-17T08:34:02.475 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:02 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:34:02.453Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-09-17T08:34:02.475 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:02 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:34:02.453Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=16.024µs wal_replay_duration=1.016054ms wbl_replay_duration=153ns total_replay_duration=1.048799ms 2024-09-17T08:34:02.475 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:02 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:34:02.455Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-17T08:34:02.475 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:02 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:34:02.455Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-17T08:34:02.475 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:02 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:34:02.455Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-17T08:34:02.475 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:02 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:34:02.473Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=18.309395ms db_storage=710ns remote_storage=1.18µs web_handler=372ns query_engine=478ns scrape=675.319µs scrape_sd=88.044µs notify=13.869µs notify_sd=8.425µs rules=17.22666ms tracing=8.016µs 2024-09-17T08:34:02.475 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:02 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:34:02.473Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-17T08:34:02.475 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:02 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:34:02.473Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-17T08:34:02.895 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:02 smithi170 bash[67404]: 993a3bf4ddb66824af98c86898194a7f16b2386a8d7937a9a363f1aa6fefb0bb 2024-09-17T08:34:02.895 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:34:02 smithi170 systemd[1]: Started Ceph prometheus.a for 7862d192-74ce-11ef-bceb-c7b262605968. 2024-09-17T08:34:02.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:02 smithi170 ceph-mon[33271]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:02.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:02 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:34:02.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:02 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:34:02.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:02 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-17T08:34:02.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:02 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi118.front.sepia.ceph.com:9093"}]: dispatch 2024-09-17T08:34:02.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:02 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:34:02.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:02 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-17T08:34:02.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:02 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi170.front.sepia.ceph.com:9095"}]: dispatch 2024-09-17T08:34:02.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:02 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:34:02.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:02 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-17T08:34:02.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:02 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi170.front.sepia.ceph.com:3000"}]: dispatch 2024-09-17T08:34:02.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:02 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:34:02.897 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:02 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:34:03.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:02 smithi118 ceph-mon[41418]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:03.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:02 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:34:03.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:02 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:34:03.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:02 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-17T08:34:03.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:02 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi118.front.sepia.ceph.com:9093"}]: dispatch 2024-09-17T08:34:03.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:02 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:34:03.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:02 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-17T08:34:03.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:02 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi170.front.sepia.ceph.com:9095"}]: dispatch 2024-09-17T08:34:03.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:02 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:34:03.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:02 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-17T08:34:03.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:02 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi170.front.sepia.ceph.com:3000"}]: dispatch 2024-09-17T08:34:03.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:02 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:34:03.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:02 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:34:03.028 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:34:02 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: [17/Sep/2024:08:34:02] ENGINE Bus STOPPING 2024-09-17T08:34:03.028 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:34:02 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: [17/Sep/2024:08:34:02] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-17T08:34:03.028 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:34:02 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: [17/Sep/2024:08:34:02] ENGINE Bus STOPPED 2024-09-17T08:34:03.029 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:34:02 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: [17/Sep/2024:08:34:02] ENGINE Bus STARTING 2024-09-17T08:34:03.029 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:34:02 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: [17/Sep/2024:08:34:02] ENGINE Serving on http://:::9283 2024-09-17T08:34:03.029 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:34:02 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: [17/Sep/2024:08:34:02] ENGINE Bus STARTED 2024-09-17T08:34:03.029 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:34:02 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: [17/Sep/2024:08:34:02] ENGINE Bus STOPPING 2024-09-17T08:34:03.029 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:02 smithi118 ceph-mon[29491]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:03.029 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:02 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:34:03.029 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:02 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:34:03.029 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:02 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-17T08:34:03.029 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:02 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi118.front.sepia.ceph.com:9093"}]: dispatch 2024-09-17T08:34:03.029 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:02 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:34:03.029 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:02 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-17T08:34:03.029 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:02 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi170.front.sepia.ceph.com:9095"}]: dispatch 2024-09-17T08:34:03.030 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:02 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:34:03.030 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:02 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-17T08:34:03.030 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:02 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi170.front.sepia.ceph.com:3000"}]: dispatch 2024-09-17T08:34:03.030 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:02 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:34:03.030 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:02 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:34:03.527 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:34:03 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: [17/Sep/2024:08:34:03] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-17T08:34:03.528 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:34:03 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: [17/Sep/2024:08:34:03] ENGINE Bus STOPPED 2024-09-17T08:34:03.528 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:34:03 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: [17/Sep/2024:08:34:03] ENGINE Bus STARTING 2024-09-17T08:34:03.528 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:34:03 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: [17/Sep/2024:08:34:03] ENGINE Serving on http://:::9283 2024-09-17T08:34:03.528 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:34:03 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: [17/Sep/2024:08:34:03] ENGINE Bus STARTED 2024-09-17T08:34:03.528 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:34:03 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: [17/Sep/2024:08:34:03] ENGINE Bus STOPPING 2024-09-17T08:34:03.981 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:03 smithi118 ceph-mon[29491]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-17T08:34:03.982 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:03 smithi118 ceph-mon[29491]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi118.front.sepia.ceph.com:9093"}]: dispatch 2024-09-17T08:34:03.982 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:03 smithi118 ceph-mon[29491]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-17T08:34:03.982 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:03 smithi118 ceph-mon[29491]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi170.front.sepia.ceph.com:9095"}]: dispatch 2024-09-17T08:34:03.982 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:03 smithi118 ceph-mon[29491]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-17T08:34:03.982 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:03 smithi118 ceph-mon[29491]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi170.front.sepia.ceph.com:3000"}]: dispatch 2024-09-17T08:34:03.982 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:34:03 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: [17/Sep/2024:08:34:03] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-17T08:34:03.982 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:34:03 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: [17/Sep/2024:08:34:03] ENGINE Bus STOPPED 2024-09-17T08:34:03.983 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:34:03 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: [17/Sep/2024:08:34:03] ENGINE Bus STARTING 2024-09-17T08:34:03.983 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:34:03 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: [17/Sep/2024:08:34:03] ENGINE Serving on http://:::9283 2024-09-17T08:34:03.983 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:03 smithi118 ceph-mon[41418]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-17T08:34:03.983 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:03 smithi118 ceph-mon[41418]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi118.front.sepia.ceph.com:9093"}]: dispatch 2024-09-17T08:34:03.983 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:03 smithi118 ceph-mon[41418]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-17T08:34:03.983 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:03 smithi118 ceph-mon[41418]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi170.front.sepia.ceph.com:9095"}]: dispatch 2024-09-17T08:34:03.983 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:03 smithi118 ceph-mon[41418]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-17T08:34:03.983 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:03 smithi118 ceph-mon[41418]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi170.front.sepia.ceph.com:3000"}]: dispatch 2024-09-17T08:34:04.133 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:03 smithi170 ceph-mon[33271]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-17T08:34:04.133 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:03 smithi170 ceph-mon[33271]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi118.front.sepia.ceph.com:9093"}]: dispatch 2024-09-17T08:34:04.133 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:03 smithi170 ceph-mon[33271]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-17T08:34:04.133 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:03 smithi170 ceph-mon[33271]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi170.front.sepia.ceph.com:9095"}]: dispatch 2024-09-17T08:34:04.133 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:03 smithi170 ceph-mon[33271]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-17T08:34:04.134 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:03 smithi170 ceph-mon[33271]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi170.front.sepia.ceph.com:3000"}]: dispatch 2024-09-17T08:34:04.270 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:34:03 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: [17/Sep/2024:08:34:03] ENGINE Bus STARTED 2024-09-17T08:34:04.780 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:04 smithi118 ceph-mon[29491]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:04.780 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:04 smithi118 ceph-mon[41418]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:04.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:04 smithi170 ceph-mon[33271]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:06.620 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:34:06.879 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:06 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:34:06.879 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:06 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:34:06.879 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:06 smithi170 ceph-mon[33271]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:06.943 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:34:06.943 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:34:06.943 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:34:06.943 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:06 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:34:06.943 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:06 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:34:06.944 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:06 smithi118 ceph-mon[29491]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:06.944 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:06 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:34:06.944 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:06 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:34:06.944 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:06 smithi118 ceph-mon[41418]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:07.602 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:34:07 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a[79253]: ts=2024-09-17T08:34:07.351Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002715718s 2024-09-17T08:34:07.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:07 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/1857426280' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:34:08.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:07 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/1857426280' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:34:08.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:07 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/1857426280' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:34:08.544 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:34:08 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:34:08.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:08 smithi170 ceph-mon[33271]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:09.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:08 smithi118 ceph-mon[29491]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:09.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:08 smithi118 ceph-mon[41418]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:10.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:09 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:34:10.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:09 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:34:10.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:09 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:34:10.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:09 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:34:10.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:09 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:34:10.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:09 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:34:10.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:09 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:34:10.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:09 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:34:10.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:09 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:34:10.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:09 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:34:10.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:09 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:34:10.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:09 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:34:10.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:10 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:34:10.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:10 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:34:10.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:10 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:34:10.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:10 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:34:10.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:10 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:34:10.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:10 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:34:11.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:11 smithi118 ceph-mon[29491]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:11.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:11 smithi118 ceph-mon[41418]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:11.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:11 smithi170 ceph-mon[33271]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:11.945 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:34:12.272 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:34:12.272 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:34:12.272 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:34:13.381 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:13 smithi118 ceph-mon[29491]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:13.381 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:13 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/2917285165' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:34:13.381 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:13 smithi118 ceph-mon[41418]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:13.381 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:13 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/2917285165' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:34:13.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:13 smithi170 ceph-mon[33271]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:13.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:13 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/2917285165' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:34:15.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:15 smithi170 ceph-mon[33271]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:15.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:15 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:34:15.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:15 smithi118 ceph-mon[29491]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:15.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:15 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:34:15.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:15 smithi118 ceph-mon[41418]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:15.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:15 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:34:17.274 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:34:17.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:17 smithi170 ceph-mon[33271]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:17.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:17 smithi118 ceph-mon[29491]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:17.527 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:34:17 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:34:17] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:34:17.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:17 smithi118 ceph-mon[41418]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:17.597 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:34:17.597 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:34:17.597 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:34:18.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:18 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/3291359736' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:34:18.395 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:34:18 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:34:18.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:18 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3291359736' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:34:18.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:18 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/3291359736' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:34:19.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:19 smithi170 ceph-mon[33271]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:19.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:19 smithi118 ceph-mon[29491]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:19.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:19 smithi118 ceph-mon[41418]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:20.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:20 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:34:20.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:20 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:34:20.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:20 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:34:21.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:21 smithi170 ceph-mon[33271]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:21.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:21 smithi118 ceph-mon[29491]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:21.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:21 smithi118 ceph-mon[41418]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:22.599 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:34:22.922 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:34:22.923 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:34:22.923 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:34:23.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:23 smithi118 ceph-mon[29491]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:23.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:23 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/2156634509' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:34:23.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:23 smithi118 ceph-mon[41418]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:23.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:23 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/2156634509' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:34:23.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:23 smithi170 ceph-mon[33271]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:23.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:23 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/2156634509' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:34:25.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:25 smithi118 ceph-mon[29491]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:25.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:25 smithi118 ceph-mon[41418]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:25.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:25 smithi170 ceph-mon[33271]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:27.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:27 smithi118 ceph-mon[29491]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:27.528 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:34:27 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:34:27] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:34:27.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:27 smithi118 ceph-mon[41418]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:27.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:27 smithi170 ceph-mon[33271]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:27.924 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:34:28.253 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:34:28.253 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:34:28.253 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:34:28.645 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:34:28 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:34:29.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:29 smithi170 ceph-mon[33271]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:29.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:29 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/2052073595' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:34:29.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:29 smithi118 ceph-mon[29491]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:29.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:29 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/2052073595' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:34:29.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:29 smithi118 ceph-mon[41418]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:29.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:29 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/2052073595' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:34:30.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:30 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:34:30.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:30 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:34:30.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:30 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:34:30.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:30 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:34:30.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:30 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:34:30.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:30 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:34:31.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:31 smithi118 ceph-mon[29491]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:31.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:31 smithi118 ceph-mon[41418]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:31.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:31 smithi170 ceph-mon[33271]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:33.255 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:34:33.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:33 smithi118 ceph-mon[29491]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:33.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:33 smithi118 ceph-mon[41418]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:33.584 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:34:33.585 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:34:33.585 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:34:33.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:33 smithi170 ceph-mon[33271]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:34.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:34 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/98088184' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:34:34.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:34 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/98088184' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:34:34.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:34 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/98088184' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:34:35.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:35 smithi118 ceph-mon[29491]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:35.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:35 smithi118 ceph-mon[41418]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:35.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:35 smithi170 ceph-mon[33271]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:37.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:37 smithi118 ceph-mon[29491]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:37.527 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:34:37 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:34:37] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:34:37.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:37 smithi118 ceph-mon[41418]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:37.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:37 smithi170 ceph-mon[33271]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:38.587 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:34:38.645 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:34:38 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:34:38.913 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:34:38.913 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:34:38.913 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:34:39.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:39 smithi118 ceph-mon[41418]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:39.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:39 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/1781912697' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:34:39.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:39 smithi118 ceph-mon[29491]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:39.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:39 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/1781912697' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:34:39.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:39 smithi170 ceph-mon[33271]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:39.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:39 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/1781912697' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:34:40.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:40 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:34:40.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:40 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:34:40.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:40 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:34:41.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:41 smithi118 ceph-mon[29491]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:41.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:41 smithi118 ceph-mon[41418]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:41.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:41 smithi170 ceph-mon[33271]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:43.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:43 smithi118 ceph-mon[29491]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:43.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:43 smithi118 ceph-mon[41418]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:43.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:43 smithi170 ceph-mon[33271]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:43.915 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:34:44.245 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:34:44.245 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:34:44.245 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:34:45.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:45 smithi118 ceph-mon[29491]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:45.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:45 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:34:45.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:45 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/1744842664' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:34:45.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:45 smithi118 ceph-mon[41418]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:45.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:45 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:34:45.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:45 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/1744842664' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:34:45.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:45 smithi170 ceph-mon[33271]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:45.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:45 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:34:45.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:45 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/1744842664' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:34:47.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:47 smithi118 ceph-mon[29491]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:47.527 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:34:47 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:34:47] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:34:47.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:47 smithi118 ceph-mon[41418]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:47.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:47 smithi170 ceph-mon[33271]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:48.645 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:34:48 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:34:49.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:48 smithi170 ceph-mon[33271]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:49.248 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:34:49.248 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:48 smithi118 ceph-mon[29491]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:49.249 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:48 smithi118 ceph-mon[41418]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:49.575 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:34:49.575 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:34:49.575 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:34:50.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:49 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:34:50.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:49 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/3692038561' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:34:50.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:49 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:34:50.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:49 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3692038561' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:34:50.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:49 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:34:50.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:49 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/3692038561' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:34:51.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:50 smithi170 ceph-mon[33271]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:51.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:50 smithi118 ceph-mon[29491]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:51.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:50 smithi118 ceph-mon[41418]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:53.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:53 smithi170 ceph-mon[33271]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:53.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:53 smithi118 ceph-mon[29491]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:53.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:53 smithi118 ceph-mon[41418]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:54.577 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:34:54.897 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:34:54.898 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:34:54.898 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:34:55.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:55 smithi118 ceph-mon[29491]: pgmap v68: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:55.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:55 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3856128828' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:34:55.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:55 smithi118 ceph-mon[41418]: pgmap v68: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:55.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:55 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/3856128828' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:34:55.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:55 smithi170 ceph-mon[33271]: pgmap v68: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:55.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:55 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/3856128828' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:34:57.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:57 smithi170 ceph-mon[33271]: pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:57.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:57 smithi118 ceph-mon[29491]: pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:57.527 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:34:57 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:34:57] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:34:57.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:57 smithi118 ceph-mon[41418]: pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:58.645 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:34:58 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:34:59.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:34:59 smithi170 ceph-mon[33271]: pgmap v70: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:59.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:34:59 smithi118 ceph-mon[29491]: pgmap v70: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:59.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:34:59 smithi118 ceph-mon[41418]: pgmap v70: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:59.899 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:35:00.231 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:35:00.231 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:35:00.231 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:35:00.231 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:00 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:35:00.231 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:00 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:35:00.232 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:00 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:35:00.232 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:00 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:35:00.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:00 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:35:00.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:00 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:35:01.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:01 smithi118 ceph-mon[29491]: pgmap v71: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:35:01.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:01 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3767611447' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:35:01.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:01 smithi118 ceph-mon[41418]: pgmap v71: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:35:01.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:01 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/3767611447' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:35:01.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:01 smithi170 ceph-mon[33271]: pgmap v71: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:35:01.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:01 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/3767611447' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:35:03.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:03 smithi118 ceph-mon[41418]: pgmap v72: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:03.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:03 smithi118 ceph-mon[29491]: pgmap v72: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:03.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:03 smithi170 ceph-mon[33271]: pgmap v72: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:05.233 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:35:05.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:05 smithi118 ceph-mon[29491]: pgmap v73: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:05.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:05 smithi118 ceph-mon[41418]: pgmap v73: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:05.562 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:35:05.562 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:35:05.562 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:35:05.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:05 smithi170 ceph-mon[33271]: pgmap v73: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:06.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:06 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/1235500961' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:35:06.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:06 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/1235500961' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:35:06.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:06 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/1235500961' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:35:07.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:07 smithi118 ceph-mon[29491]: pgmap v74: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:35:07.528 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:35:07 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:35:07] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:35:07.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:07 smithi118 ceph-mon[41418]: pgmap v74: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:35:07.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:07 smithi170 ceph-mon[33271]: pgmap v74: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:35:08.645 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:35:08 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:35:09.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:09 smithi118 ceph-mon[29491]: pgmap v75: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:09.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:09 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:35:09.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:09 smithi118 ceph-mon[41418]: pgmap v75: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:09.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:09 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:35:09.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:09 smithi170 ceph-mon[33271]: pgmap v75: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:09.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:09 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:35:10.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:10 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:35:10.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:10 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:35:10.563 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:35:10.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:10 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:35:10.884 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:35:10.884 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:35:10.884 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:35:11.500 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:11 smithi170 ceph-mon[33271]: pgmap v76: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:35:11.500 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:11 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/3344378009' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:35:11.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:11 smithi118 ceph-mon[29491]: pgmap v76: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:35:11.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:11 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3344378009' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:35:11.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:11 smithi118 ceph-mon[41418]: pgmap v76: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:35:11.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:11 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/3344378009' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:35:13.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:12 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:35:13.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:12 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:35:13.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:12 smithi170 ceph-mon[33271]: pgmap v77: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:13.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:12 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:35:13.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:12 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:35:13.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:12 smithi118 ceph-mon[29491]: pgmap v77: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:13.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:12 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:35:13.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:12 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:35:13.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:12 smithi118 ceph-mon[41418]: pgmap v77: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:14.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:14 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:35:14.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:14 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:35:14.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:14 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:35:14.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:14 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:35:14.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:14 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:35:14.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:14 smithi170 ceph-mon[33271]: pgmap v78: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:14.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:14 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:35:14.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:14 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:35:14.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:14 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:35:14.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:14 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:35:14.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:14 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:35:14.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:14 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:35:14.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:14 smithi118 ceph-mon[29491]: pgmap v78: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:14.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:14 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:35:14.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:14 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:35:14.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:14 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:35:14.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:14 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:35:14.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:14 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:35:14.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:14 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:35:14.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:14 smithi118 ceph-mon[41418]: pgmap v78: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:14.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:14 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:35:15.886 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:35:16.210 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:35:16.210 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:35:16.210 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:35:17.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:17 smithi170 ceph-mon[33271]: pgmap v79: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:35:17.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:17 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/955330002' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:35:17.527 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:35:17 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:35:17] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:35:17.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:17 smithi118 ceph-mon[41418]: pgmap v79: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:35:17.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:17 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/955330002' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:35:17.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:17 smithi118 ceph-mon[29491]: pgmap v79: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:35:17.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:17 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/955330002' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:35:18.645 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:35:18 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:35:19.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:19 smithi170 ceph-mon[33271]: pgmap v80: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:19.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:19 smithi118 ceph-mon[29491]: pgmap v80: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:19.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:19 smithi118 ceph-mon[41418]: pgmap v80: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:20.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:20 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:35:20.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:20 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:35:20.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:20 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:35:21.211 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:35:21.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:21 smithi118 ceph-mon[29491]: pgmap v81: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:35:21.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:21 smithi118 ceph-mon[41418]: pgmap v81: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:35:21.537 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:35:21.547 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:35:21.547 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:35:21.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:21 smithi170 ceph-mon[33271]: pgmap v81: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:35:22.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:22 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/1442324206' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:35:22.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:22 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/1442324206' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:35:22.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:22 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/1442324206' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:35:23.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:23 smithi118 ceph-mon[29491]: pgmap v82: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:23.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:23 smithi118 ceph-mon[41418]: pgmap v82: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:23.648 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:23 smithi170 ceph-mon[33271]: pgmap v82: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:25.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:25 smithi118 ceph-mon[29491]: pgmap v83: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:25.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:25 smithi118 ceph-mon[41418]: pgmap v83: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:25.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:25 smithi170 ceph-mon[33271]: pgmap v83: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:26.539 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:35:26.865 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:35:26.865 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:35:26.866 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:35:27.527 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:35:27 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:35:27] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:35:27.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:27 smithi118 ceph-mon[29491]: pgmap v84: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:35:27.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:27 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/634748784' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:35:27.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:27 smithi118 ceph-mon[41418]: pgmap v84: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:35:27.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:27 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/634748784' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:35:27.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:27 smithi170 ceph-mon[33271]: pgmap v84: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:35:27.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:27 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/634748784' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:35:28.645 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:35:28 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:35:29.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:29 smithi118 ceph-mon[29491]: pgmap v85: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:29.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:29 smithi118 ceph-mon[41418]: pgmap v85: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:29.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:29 smithi170 ceph-mon[33271]: pgmap v85: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:30.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:30 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:35:30.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:30 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:35:30.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:30 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:35:30.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:30 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:35:30.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:30 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:35:30.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:30 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:35:31.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:31 smithi118 ceph-mon[29491]: pgmap v86: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:35:31.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:31 smithi118 ceph-mon[41418]: pgmap v86: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:35:31.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:31 smithi170 ceph-mon[33271]: pgmap v86: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:35:31.867 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:35:32.194 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:35:32.195 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:35:32.195 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:35:33.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:33 smithi118 ceph-mon[29491]: pgmap v87: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:33.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:33 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/678402693' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:35:33.429 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:33 smithi118 ceph-mon[41418]: pgmap v87: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:33.429 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:33 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/678402693' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:35:33.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:33 smithi170 ceph-mon[33271]: pgmap v87: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:33.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:33 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/678402693' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:35:35.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:35 smithi118 ceph-mon[29491]: pgmap v88: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:35.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:35 smithi118 ceph-mon[41418]: pgmap v88: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:35.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:35 smithi170 ceph-mon[33271]: pgmap v88: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:37.197 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:35:37.515 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:37 smithi118 ceph-mon[29491]: pgmap v89: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:35:37.515 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:35:37 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:35:37] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:35:37.515 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:35:37.516 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:35:37.516 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:35:37.516 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:37 smithi118 ceph-mon[41418]: pgmap v89: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:35:37.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:37 smithi170 ceph-mon[33271]: pgmap v89: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:35:38.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:38 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3472105866' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:35:38.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:38 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/3472105866' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:35:38.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:38 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/3472105866' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:35:38.645 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:35:38 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:35:39.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:39 smithi118 ceph-mon[29491]: pgmap v90: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:39.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:39 smithi118 ceph-mon[41418]: pgmap v90: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:39.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:39 smithi170 ceph-mon[33271]: pgmap v90: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:40.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:40 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:35:40.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:40 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:35:40.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:40 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:35:41.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:41 smithi118 ceph-mon[29491]: pgmap v91: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:35:41.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:41 smithi118 ceph-mon[41418]: pgmap v91: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:35:41.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:41 smithi170 ceph-mon[33271]: pgmap v91: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:35:42.517 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:35:42.843 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:35:42.843 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:35:42.844 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:35:43.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:43 smithi118 ceph-mon[29491]: pgmap v92: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:43.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:43 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3057006523' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:35:43.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:43 smithi118 ceph-mon[41418]: pgmap v92: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:43.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:43 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/3057006523' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:35:43.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:43 smithi170 ceph-mon[33271]: pgmap v92: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:43.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:43 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/3057006523' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:35:45.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:45 smithi118 ceph-mon[41418]: pgmap v93: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:45.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:45 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:35:45.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:45 smithi118 ceph-mon[29491]: pgmap v93: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:45.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:45 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:35:45.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:45 smithi170 ceph-mon[33271]: pgmap v93: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:45.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:45 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:35:47.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:47 smithi118 ceph-mon[29491]: pgmap v94: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:35:47.527 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:35:47 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:35:47] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:35:47.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:47 smithi118 ceph-mon[41418]: pgmap v94: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:35:47.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:47 smithi170 ceph-mon[33271]: pgmap v94: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:35:47.846 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:35:48.169 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:35:48.169 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:35:48.169 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:35:48.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:48 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3263282783' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:35:48.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:48 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/3263282783' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:35:48.546 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:48 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/3263282783' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:35:48.546 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:35:48 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:35:49.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:49 smithi118 ceph-mon[29491]: pgmap v95: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:49.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:49 smithi118 ceph-mon[41418]: pgmap v95: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:49.571 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:49 smithi170 ceph-mon[33271]: pgmap v95: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:50.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:50 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:35:50.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:50 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:35:50.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:50 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:35:51.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:51 smithi118 ceph-mon[29491]: pgmap v96: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:35:51.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:51 smithi118 ceph-mon[41418]: pgmap v96: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:35:51.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:51 smithi170 ceph-mon[33271]: pgmap v96: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:35:53.171 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:35:53.496 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:35:53.496 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:35:53.496 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:35:53.496 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:53 smithi118 ceph-mon[29491]: pgmap v97: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:53.497 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:53 smithi118 ceph-mon[41418]: pgmap v97: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:53.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:53 smithi170 ceph-mon[33271]: pgmap v97: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:54.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:54 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/554130109' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:35:54.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:54 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/554130109' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:35:54.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:54 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/554130109' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:35:55.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:55 smithi118 ceph-mon[29491]: pgmap v98: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:55.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:55 smithi118 ceph-mon[41418]: pgmap v98: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:55.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:55 smithi170 ceph-mon[33271]: pgmap v98: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:57.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:57 smithi118 ceph-mon[29491]: pgmap v99: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:35:57.527 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:35:57 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:35:57] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:35:57.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:57 smithi118 ceph-mon[41418]: pgmap v99: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:35:57.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:57 smithi170 ceph-mon[33271]: pgmap v99: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:35:58.498 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:35:58.645 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:35:58 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:35:58.823 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:35:58.823 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:35:58.823 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:35:59.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:59 smithi118 ceph-mon[29491]: pgmap v100: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:59.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:35:59 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3393703457' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:35:59.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:59 smithi118 ceph-mon[41418]: pgmap v100: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:59.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:35:59 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/3393703457' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:35:59.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:59 smithi170 ceph-mon[33271]: pgmap v100: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:35:59.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:35:59 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/3393703457' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:36:00.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:00 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:36:00.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:00 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:36:00.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:00 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:36:00.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:00 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:36:00.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:00 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:36:00.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:00 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:36:01.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:01 smithi118 ceph-mon[29491]: pgmap v101: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:36:01.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:01 smithi118 ceph-mon[41418]: pgmap v101: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:36:01.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:01 smithi170 ceph-mon[33271]: pgmap v101: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:36:03.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:03 smithi118 ceph-mon[29491]: pgmap v102: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:03.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:03 smithi118 ceph-mon[41418]: pgmap v102: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:03.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:03 smithi170 ceph-mon[33271]: pgmap v102: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:03.825 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:36:04.149 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:36:04.149 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:36:04.149 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:36:04.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:04 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3594151988' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:36:04.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:04 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/3594151988' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:36:04.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:04 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/3594151988' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:36:05.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:05 smithi118 ceph-mon[29491]: pgmap v103: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:05.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:05 smithi118 ceph-mon[41418]: pgmap v103: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:05.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:05 smithi170 ceph-mon[33271]: pgmap v103: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:07.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:07 smithi118 ceph-mon[41418]: pgmap v104: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:36:07.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:07 smithi118 ceph-mon[29491]: pgmap v104: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:36:07.528 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:36:07 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:36:07] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:36:07.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:07 smithi170 ceph-mon[33271]: pgmap v104: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:36:08.895 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:36:08 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:36:09.150 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:36:09.482 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:36:09.482 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:36:09.482 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:36:09.483 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:09 smithi118 ceph-mon[41418]: pgmap v105: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:09.483 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:09 smithi118 ceph-mon[29491]: pgmap v105: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:09.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:09 smithi170 ceph-mon[33271]: pgmap v105: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:10.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:10 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:36:10.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:10 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3262648451' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:36:10.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:10 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:36:10.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:10 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/3262648451' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:36:10.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:10 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:36:10.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:10 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/3262648451' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:36:11.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:11 smithi118 ceph-mon[29491]: pgmap v106: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:36:11.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:11 smithi118 ceph-mon[41418]: pgmap v106: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:36:11.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:11 smithi170 ceph-mon[33271]: pgmap v106: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:36:12.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:12 smithi170 ceph-mon[33271]: pgmap v107: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:12.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:12 smithi118 ceph-mon[29491]: pgmap v107: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:12.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:12 smithi118 ceph-mon[41418]: pgmap v107: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:14.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:13 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:36:14.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:13 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:36:14.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:13 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:36:14.484 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:36:14.802 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:36:14.802 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:36:14.802 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:36:15.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:14 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:36:15.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:14 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:36:15.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:14 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:36:15.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:14 smithi118 ceph-mon[29491]: pgmap v108: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:15.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:14 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:36:15.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:14 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:36:15.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:14 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:36:15.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:14 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:36:15.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:14 smithi118 ceph-mon[41418]: pgmap v108: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:15.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:14 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:36:15.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:14 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:36:15.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:14 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:36:15.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:14 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:36:15.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:14 smithi170 ceph-mon[33271]: pgmap v108: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:15.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:14 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:36:16.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:15 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/1041072503' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:36:16.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:15 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/1041072503' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:36:16.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:15 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/1041072503' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:36:17.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:16 smithi118 ceph-mon[29491]: pgmap v109: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:36:17.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:16 smithi118 ceph-mon[41418]: pgmap v109: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:36:17.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:16 smithi170 ceph-mon[33271]: pgmap v109: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:36:17.527 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:36:17 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:36:17] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:36:18.895 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:36:18 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:36:19.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:19 smithi118 ceph-mon[29491]: pgmap v110: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:19.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:19 smithi118 ceph-mon[41418]: pgmap v110: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:19.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:19 smithi170 ceph-mon[33271]: pgmap v110: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:19.804 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:36:20.131 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:36:20.131 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:36:20.131 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:36:20.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:20 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:36:20.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:20 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/2010473047' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:36:20.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:20 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:36:20.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:20 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/2010473047' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:36:20.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:20 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:36:20.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:20 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/2010473047' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:36:21.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:21 smithi118 ceph-mon[29491]: pgmap v111: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:36:21.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:21 smithi118 ceph-mon[41418]: pgmap v111: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:36:21.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:21 smithi170 ceph-mon[33271]: pgmap v111: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:36:23.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:23 smithi118 ceph-mon[29491]: pgmap v112: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:23.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:23 smithi118 ceph-mon[41418]: pgmap v112: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:23.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:23 smithi170 ceph-mon[33271]: pgmap v112: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:25.133 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:36:25.461 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:36:25.461 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:36:25.461 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:36:25.462 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:25 smithi118 ceph-mon[29491]: pgmap v113: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:25.462 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:25 smithi118 ceph-mon[41418]: pgmap v113: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:25.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:25 smithi170 ceph-mon[33271]: pgmap v113: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:26.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:26 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/2992796848' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:36:26.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:26 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/2992796848' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:36:26.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:26 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/2992796848' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:36:27.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:27 smithi118 ceph-mon[29491]: pgmap v114: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:36:27.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:27 smithi118 ceph-mon[41418]: pgmap v114: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:36:27.528 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:36:27 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:36:27] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:36:27.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:27 smithi170 ceph-mon[33271]: pgmap v114: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:36:28.895 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:36:28 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:36:29.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:29 smithi118 ceph-mon[29491]: pgmap v115: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:29.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:29 smithi118 ceph-mon[41418]: pgmap v115: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:29.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:29 smithi170 ceph-mon[33271]: pgmap v115: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:30.463 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:36:30.464 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:30 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:36:30.464 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:30 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:36:30.464 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:30 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:36:30.464 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:30 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:36:30.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:30 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:36:30.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:30 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:36:30.792 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:36:30.792 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:36:30.792 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:36:31.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:31 smithi118 ceph-mon[29491]: pgmap v116: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:36:31.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:31 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/1720248571' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:36:31.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:31 smithi118 ceph-mon[41418]: pgmap v116: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:36:31.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:31 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/1720248571' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:36:31.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:31 smithi170 ceph-mon[33271]: pgmap v116: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:36:31.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:31 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/1720248571' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:36:33.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:33 smithi118 ceph-mon[29491]: pgmap v117: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:33.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:33 smithi118 ceph-mon[41418]: pgmap v117: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:33.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:33 smithi170 ceph-mon[33271]: pgmap v117: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:35.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:35 smithi118 ceph-mon[29491]: pgmap v118: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:35.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:35 smithi118 ceph-mon[41418]: pgmap v118: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:35.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:35 smithi170 ceph-mon[33271]: pgmap v118: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:35.794 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:36:36.129 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:36:36.129 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:36:36.129 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:36:36.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:36 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3516087056' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:36:36.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:36 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/3516087056' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:36:36.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:36 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/3516087056' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:36:37.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:37 smithi118 ceph-mon[29491]: pgmap v119: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:36:37.527 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:36:37 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:36:37] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:36:37.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:37 smithi118 ceph-mon[41418]: pgmap v119: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:36:37.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:37 smithi170 ceph-mon[33271]: pgmap v119: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:36:38.895 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:36:38 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:36:39.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:39 smithi118 ceph-mon[29491]: pgmap v120: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:39.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:39 smithi118 ceph-mon[41418]: pgmap v120: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:39.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:39 smithi170 ceph-mon[33271]: pgmap v120: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:40.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:40 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:36:40.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:40 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:36:40.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:40 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:36:41.129 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:36:41.457 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:36:41.457 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:41 smithi118 ceph-mon[29491]: pgmap v121: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:36:41.458 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:36:41.458 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:36:41.458 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:41 smithi118 ceph-mon[41418]: pgmap v121: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:36:41.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:41 smithi170 ceph-mon[33271]: pgmap v121: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:36:42.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:42 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/1314130670' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:36:42.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:42 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/1314130670' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:36:42.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:42 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/1314130670' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:36:43.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:43 smithi118 ceph-mon[29491]: pgmap v122: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:43.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:43 smithi118 ceph-mon[41418]: pgmap v122: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:43.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:43 smithi170 ceph-mon[33271]: pgmap v122: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:45.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:45 smithi118 ceph-mon[29491]: pgmap v123: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:45.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:45 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:36:45.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:45 smithi118 ceph-mon[41418]: pgmap v123: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:45.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:45 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:36:45.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:45 smithi170 ceph-mon[33271]: pgmap v123: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:45.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:45 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:36:46.459 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:36:46.786 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:36:46.787 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:36:46.787 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:36:47.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:47 smithi118 ceph-mon[29491]: pgmap v124: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:36:47.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:47 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/651239711' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:36:47.527 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:36:47 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:36:47] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:36:47.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:47 smithi118 ceph-mon[41418]: pgmap v124: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:36:47.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:47 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/651239711' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:36:47.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:47 smithi170 ceph-mon[33271]: pgmap v124: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:36:47.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:47 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/651239711' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:36:48.895 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:36:48 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:36:49.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:49 smithi118 ceph-mon[29491]: pgmap v125: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:49.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:49 smithi118 ceph-mon[41418]: pgmap v125: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:49.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:49 smithi170 ceph-mon[33271]: pgmap v125: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:50.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:50 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:36:50.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:50 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:36:50.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:50 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:36:51.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:51 smithi118 ceph-mon[29491]: pgmap v126: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:36:51.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:51 smithi118 ceph-mon[41418]: pgmap v126: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:36:51.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:51 smithi170 ceph-mon[33271]: pgmap v126: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:36:51.788 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:36:52.117 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:36:52.117 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:36:52.117 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:36:52.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:52 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/2504472934' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:36:52.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:52 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/2504472934' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:36:52.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:52 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/2504472934' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:36:53.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:53 smithi118 ceph-mon[29491]: pgmap v127: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:53.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:53 smithi118 ceph-mon[41418]: pgmap v127: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:53.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:53 smithi170 ceph-mon[33271]: pgmap v127: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:55.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:55 smithi118 ceph-mon[29491]: pgmap v128: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:55.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:55 smithi118 ceph-mon[41418]: pgmap v128: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:55.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:55 smithi170 ceph-mon[33271]: pgmap v128: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:57.119 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:36:57.442 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:57 smithi118 ceph-mon[29491]: pgmap v129: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:36:57.442 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:36:57.442 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:36:57.442 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:36:57.443 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:36:57 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:36:57] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:36:57.443 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:57 smithi118 ceph-mon[41418]: pgmap v129: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:36:57.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:57 smithi170 ceph-mon[33271]: pgmap v129: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:36:58.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:58 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/2593690979' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:36:58.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:58 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/2593690979' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:36:58.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:58 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/2593690979' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:36:58.645 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:36:58 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:36:59.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:36:59 smithi118 ceph-mon[29491]: pgmap v130: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:59.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:36:59 smithi118 ceph-mon[41418]: pgmap v130: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:36:59.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:36:59 smithi170 ceph-mon[33271]: pgmap v130: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:00.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:00 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:37:00.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:00 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:37:00.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:00 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:37:00.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:00 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:37:00.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:00 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:37:00.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:00 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:37:01.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:01 smithi118 ceph-mon[29491]: pgmap v131: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:37:01.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:01 smithi118 ceph-mon[41418]: pgmap v131: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:37:01.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:01 smithi170 ceph-mon[33271]: pgmap v131: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:37:02.443 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:37:02.768 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:37:02.768 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:37:02.768 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:37:03.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:03 smithi118 ceph-mon[29491]: pgmap v132: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:03.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:03 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/2077126155' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:37:03.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:03 smithi118 ceph-mon[41418]: pgmap v132: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:03.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:03 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/2077126155' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:37:03.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:03 smithi170 ceph-mon[33271]: pgmap v132: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:03.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:03 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/2077126155' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:37:05.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:05 smithi118 ceph-mon[29491]: pgmap v133: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:05.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:05 smithi118 ceph-mon[41418]: pgmap v133: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:05.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:05 smithi170 ceph-mon[33271]: pgmap v133: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:07.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:07 smithi118 ceph-mon[29491]: pgmap v134: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:37:07.527 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:37:07 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:37:07] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:37:07.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:07 smithi118 ceph-mon[41418]: pgmap v134: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:37:07.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:07 smithi170 ceph-mon[33271]: pgmap v134: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:37:07.770 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:37:08.094 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:37:08.094 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:37:08.094 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:37:08.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:08 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/3509286545' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:37:08.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:08 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3509286545' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:37:08.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:08 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/3509286545' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:37:08.645 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:37:08 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:37:09.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:09 smithi118 ceph-mon[29491]: pgmap v135: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:09.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:09 smithi118 ceph-mon[41418]: pgmap v135: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:09.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:09 smithi170 ceph-mon[33271]: pgmap v135: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:10.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:10 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:37:10.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:10 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:37:10.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:10 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:37:11.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:11 smithi118 ceph-mon[29491]: pgmap v136: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:37:11.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:11 smithi118 ceph-mon[41418]: pgmap v136: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:37:11.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:11 smithi170 ceph-mon[33271]: pgmap v136: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:37:13.096 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:37:13.423 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== 2024-09-17T08:37:13.423 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== ']' 2024-09-17T08:37:13.423 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:37:13.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:13 smithi118 ceph-mon[29491]: pgmap v137: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:13.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:13 smithi118 ceph-mon[41418]: pgmap v137: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:13.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:13 smithi170 ceph-mon[33271]: pgmap v137: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:14.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:14 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/544831331' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:37:14.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:14 smithi170 ceph-mon[33271]: pgmap v138: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:14.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:14 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:37:14.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:14 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/544831331' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:37:14.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:14 smithi118 ceph-mon[29491]: pgmap v138: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:14.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:14 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:37:14.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:14 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/544831331' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:37:14.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:14 smithi118 ceph-mon[41418]: pgmap v138: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:14.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:14 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:37:15.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:15 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:37:15.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:15 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:37:15.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:15 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:37:15.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:15 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:15.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:15 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:37:15.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:15 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:37:15.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:15 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:37:15.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:15 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:15.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:15 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:37:15.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:15 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:37:15.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:15 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:37:15.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:15 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:17.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:17 smithi170 ceph-mon[33271]: pgmap v139: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:37:17.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:17 smithi118 ceph-mon[29491]: pgmap v139: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:37:17.527 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:37:17 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:37:17] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:37:17.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:17 smithi118 ceph-mon[41418]: pgmap v139: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:37:18.425 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.0 2024-09-17T08:37:18.752 INFO:teuthology.orchestra.run.smithi118.stdout:rotating key for osd.1 2024-09-17T08:37:18.753 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQDtPulmV780LRAAgWEy5oZSSyYx2YnTcg3E5g== 2024-09-17T08:37:18.753 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQC8PelmSQcIOhAAoocl1zIMhyS+33FBMjP9dg== == AQDtPulmV780LRAAgWEy5oZSSyYx2YnTcg3E5g== ']' 2024-09-17T08:37:18.753 INFO:teuthology.orchestra.run.smithi118.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-17T08:37:18.753 INFO:teuthology.orchestra.run.smithi118.stderr:+ echo 'rotating key for osd.1' 2024-09-17T08:37:18.753 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.1 2024-09-17T08:37:18.895 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:37:18 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:37:19.066 INFO:teuthology.orchestra.run.smithi118.stderr:+ K=AQDWPelmXvEVABAAGSOUCpNeYUMW2wtydCKEaA== 2024-09-17T08:37:19.067 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQDWPelmXvEVABAAGSOUCpNeYUMW2wtydCKEaA== 2024-09-17T08:37:19.067 INFO:teuthology.orchestra.run.smithi118.stderr:+ ceph orch daemon rotate-key osd.1 2024-09-17T08:37:19.331 INFO:teuthology.orchestra.run.smithi118.stdout:Scheduled to rotate-key osd.1 on host 'smithi118' 2024-09-17T08:37:19.331 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:19 smithi118 ceph-mon[29491]: pgmap v140: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:19.332 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:19 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/4037250812' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:37:19.332 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:19 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/2219450183' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-17T08:37:19.332 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:19 smithi118 ceph-mon[41418]: pgmap v140: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:19.332 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:19 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/4037250812' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:37:19.332 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:19 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/2219450183' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-17T08:37:19.343 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQDWPelmXvEVABAAGSOUCpNeYUMW2wtydCKEaA== == AQDWPelmXvEVABAAGSOUCpNeYUMW2wtydCKEaA== ']' 2024-09-17T08:37:19.344 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:37:19.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:19 smithi170 ceph-mon[33271]: pgmap v140: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:19.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:19 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/4037250812' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-17T08:37:19.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:19 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/2219450183' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-17T08:37:20.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:20 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:37:20.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:20 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:20.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:20 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:20.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:20 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:37:20.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:20 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:37:20.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:20 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:37:20.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:20 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:20.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:20 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]: dispatch 2024-09-17T08:37:20.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:20 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]: dispatch 2024-09-17T08:37:20.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:20 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]': finished 2024-09-17T08:37:20.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:20 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:37:20.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:20 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:20.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:20 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:20.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:20 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:37:20.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:20 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:37:20.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:20 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:37:20.529 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:20 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:20.529 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:20 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]: dispatch 2024-09-17T08:37:20.529 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:20 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]: dispatch 2024-09-17T08:37:20.529 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:20 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]': finished 2024-09-17T08:37:20.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:20 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:37:20.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:20 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:20.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:20 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:20.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:20 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:37:20.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:20 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:37:20.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:20 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:37:20.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:20 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:20.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:20 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]: dispatch 2024-09-17T08:37:20.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:20 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]: dispatch 2024-09-17T08:37:20.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:20 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]': finished 2024-09-17T08:37:21.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:21 smithi118 ceph-mon[29491]: from='client.24878 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:37:21.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:21 smithi118 ceph-mon[29491]: Schedule rotate-key daemon osd.1 2024-09-17T08:37:21.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:21 smithi118 ceph-mon[29491]: Rotating authentication key for osd.1 2024-09-17T08:37:21.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:21 smithi118 ceph-mon[29491]: Reconfiguring daemon osd.1 on smithi118 2024-09-17T08:37:21.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:21 smithi118 ceph-mon[29491]: pgmap v141: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:37:21.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:21 smithi118 ceph-mon[41418]: from='client.24878 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:37:21.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:21 smithi118 ceph-mon[41418]: Schedule rotate-key daemon osd.1 2024-09-17T08:37:21.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:21 smithi118 ceph-mon[41418]: Rotating authentication key for osd.1 2024-09-17T08:37:21.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:21 smithi118 ceph-mon[41418]: Reconfiguring daemon osd.1 on smithi118 2024-09-17T08:37:21.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:21 smithi118 ceph-mon[41418]: pgmap v141: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:37:21.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:21 smithi170 ceph-mon[33271]: from='client.24878 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:37:21.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:21 smithi170 ceph-mon[33271]: Schedule rotate-key daemon osd.1 2024-09-17T08:37:21.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:21 smithi170 ceph-mon[33271]: Rotating authentication key for osd.1 2024-09-17T08:37:21.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:21 smithi170 ceph-mon[33271]: Reconfiguring daemon osd.1 on smithi118 2024-09-17T08:37:21.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:21 smithi170 ceph-mon[33271]: pgmap v141: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:37:23.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:22 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:23.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:22 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:23.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:22 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:23.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:22 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:23.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:22 smithi118 ceph-mon[41418]: pgmap v142: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:23.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:22 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:23.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:22 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:23.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:22 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:23.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:22 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:23.028 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:22 smithi118 ceph-mon[29491]: pgmap v142: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:23.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:22 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:23.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:22 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:23.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:22 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:23.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:22 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:23.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:22 smithi170 ceph-mon[33271]: pgmap v142: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:24.346 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.1 2024-09-17T08:37:24.673 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQDWPelmXvEVABAAGSOUCpNeYUMW2wtydCKEaA== 2024-09-17T08:37:24.673 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQDWPelmXvEVABAAGSOUCpNeYUMW2wtydCKEaA== == AQDWPelmXvEVABAAGSOUCpNeYUMW2wtydCKEaA== ']' 2024-09-17T08:37:24.673 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:37:25.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:25 smithi118 ceph-mon[29491]: pgmap v143: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:25.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:25 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3198128164' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-17T08:37:25.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:25 smithi118 ceph-mon[41418]: pgmap v143: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:25.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:25 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/3198128164' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-17T08:37:25.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:25 smithi170 ceph-mon[33271]: pgmap v143: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:25.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:25 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/3198128164' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-17T08:37:27.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:27 smithi118 ceph-mon[29491]: pgmap v144: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:37:27.527 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:37:27 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:37:27] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:37:27.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:27 smithi118 ceph-mon[41418]: pgmap v144: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:37:27.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:27 smithi170 ceph-mon[33271]: pgmap v144: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:37:28.895 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:37:28 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:37:29.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:29 smithi118 ceph-mon[29491]: pgmap v145: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:29.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:29 smithi118 ceph-mon[41418]: pgmap v145: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:29.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:29 smithi170 ceph-mon[33271]: pgmap v145: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:29.675 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.1 2024-09-17T08:37:30.002 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQDWPelmXvEVABAAGSOUCpNeYUMW2wtydCKEaA== 2024-09-17T08:37:30.002 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQDWPelmXvEVABAAGSOUCpNeYUMW2wtydCKEaA== == AQDWPelmXvEVABAAGSOUCpNeYUMW2wtydCKEaA== ']' 2024-09-17T08:37:30.003 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:37:30.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:30 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:37:30.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:30 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:37:30.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:30 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/4228545441' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-17T08:37:30.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:30 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:37:30.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:30 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:37:30.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:30 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/4228545441' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-17T08:37:30.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:30 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:37:30.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:30 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:37:30.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:30 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/4228545441' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-17T08:37:31.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:31 smithi118 ceph-mon[29491]: pgmap v146: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:37:31.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:31 smithi118 ceph-mon[41418]: pgmap v146: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:37:31.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:31 smithi170 ceph-mon[33271]: pgmap v146: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:37:33.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:33 smithi118 ceph-mon[29491]: pgmap v147: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:33.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:33 smithi118 ceph-mon[41418]: pgmap v147: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:33.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:33 smithi170 ceph-mon[33271]: pgmap v147: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:35.005 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.1 2024-09-17T08:37:35.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:35 smithi118 ceph-mon[29491]: pgmap v148: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:35.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:35 smithi118 ceph-mon[41418]: pgmap v148: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:35.323 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQDWPelmXvEVABAAGSOUCpNeYUMW2wtydCKEaA== 2024-09-17T08:37:35.323 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQDWPelmXvEVABAAGSOUCpNeYUMW2wtydCKEaA== == AQDWPelmXvEVABAAGSOUCpNeYUMW2wtydCKEaA== ']' 2024-09-17T08:37:35.323 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:37:35.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:35 smithi170 ceph-mon[33271]: pgmap v148: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:36.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:36 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3763595143' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-17T08:37:36.529 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:36 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/3763595143' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-17T08:37:36.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:36 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/3763595143' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-17T08:37:37.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:37 smithi118 ceph-mon[29491]: pgmap v149: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:37:37.527 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:37:37 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:37:37] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:37:37.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:37 smithi118 ceph-mon[41418]: pgmap v149: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:37:37.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:37 smithi170 ceph-mon[33271]: pgmap v149: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:37:38.895 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:37:38 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:37:39.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:39 smithi118 ceph-mon[29491]: pgmap v150: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:39.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:39 smithi118 ceph-mon[41418]: pgmap v150: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:39.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:39 smithi170 ceph-mon[33271]: pgmap v150: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:40.325 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.1 2024-09-17T08:37:40.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:40 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:37:40.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:40 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:37:40.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:40 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:37:40.654 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQDWPelmXvEVABAAGSOUCpNeYUMW2wtydCKEaA== 2024-09-17T08:37:40.654 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQDWPelmXvEVABAAGSOUCpNeYUMW2wtydCKEaA== == AQDWPelmXvEVABAAGSOUCpNeYUMW2wtydCKEaA== ']' 2024-09-17T08:37:40.654 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:37:41.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:41 smithi118 ceph-mon[29491]: pgmap v151: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:37:41.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:41 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/2807115937' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-17T08:37:41.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:41 smithi118 ceph-mon[41418]: pgmap v151: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:37:41.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:41 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/2807115937' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-17T08:37:41.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:41 smithi170 ceph-mon[33271]: pgmap v151: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:37:41.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:41 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/2807115937' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-17T08:37:43.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:43 smithi118 ceph-mon[29491]: pgmap v152: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:43.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:43 smithi118 ceph-mon[41418]: pgmap v152: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:43.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:43 smithi170 ceph-mon[33271]: pgmap v152: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:45.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:45 smithi118 ceph-mon[29491]: pgmap v153: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:45.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:45 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:37:45.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:45 smithi118 ceph-mon[41418]: pgmap v153: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:45.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:45 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:37:45.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:45 smithi170 ceph-mon[33271]: pgmap v153: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:45.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:45 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:37:45.656 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.1 2024-09-17T08:37:45.979 INFO:teuthology.orchestra.run.smithi118.stdout:rotating key for osd.2 2024-09-17T08:37:45.980 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQC/P+lmFRyVGBAA4d9t1l0e6lx1TS4vEmrMMA== 2024-09-17T08:37:45.980 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQDWPelmXvEVABAAGSOUCpNeYUMW2wtydCKEaA== == AQC/P+lmFRyVGBAA4d9t1l0e6lx1TS4vEmrMMA== ']' 2024-09-17T08:37:45.980 INFO:teuthology.orchestra.run.smithi118.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-17T08:37:45.980 INFO:teuthology.orchestra.run.smithi118.stderr:+ echo 'rotating key for osd.2' 2024-09-17T08:37:45.980 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.2 2024-09-17T08:37:46.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:46 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/1424278979' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-17T08:37:46.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:46 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/1424278979' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-17T08:37:46.297 INFO:teuthology.orchestra.run.smithi118.stderr:+ K=AQDuPelm6EKBOhAAmJAejkSxjMM0DwWoNUdhlw== 2024-09-17T08:37:46.297 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQDuPelm6EKBOhAAmJAejkSxjMM0DwWoNUdhlw== 2024-09-17T08:37:46.297 INFO:teuthology.orchestra.run.smithi118.stderr:+ ceph orch daemon rotate-key osd.2 2024-09-17T08:37:46.562 INFO:teuthology.orchestra.run.smithi118.stdout:Scheduled to rotate-key osd.2 on host 'smithi118' 2024-09-17T08:37:46.573 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQDuPelm6EKBOhAAmJAejkSxjMM0DwWoNUdhlw== == AQDuPelm6EKBOhAAmJAejkSxjMM0DwWoNUdhlw== ']' 2024-09-17T08:37:46.573 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:37:46.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:46 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/1424278979' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-17T08:37:47.444 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:37:47 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:37:47] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:37:47.444 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:47 smithi118 ceph-mon[29491]: pgmap v154: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:37:47.444 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:47 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3613914158' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-17T08:37:47.444 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:47 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:47.444 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:47 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:47.445 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:47 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:37:47.445 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:47 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:47.445 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:47 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:47.445 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:47 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:37:47.445 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:47 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:37:47.445 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:47 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:47.445 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:47 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]: dispatch 2024-09-17T08:37:47.445 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:47 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]: dispatch 2024-09-17T08:37:47.445 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:47 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]': finished 2024-09-17T08:37:47.445 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:47 smithi118 ceph-mon[41418]: pgmap v154: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:37:47.445 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:47 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/3613914158' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-17T08:37:47.445 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:47 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:47.446 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:47 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:47.446 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:47 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:37:47.446 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:47 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:47.446 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:47 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:47.446 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:47 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:37:47.446 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:47 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:37:47.446 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:47 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:47.446 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:47 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]: dispatch 2024-09-17T08:37:47.446 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:47 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]: dispatch 2024-09-17T08:37:47.446 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:47 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]': finished 2024-09-17T08:37:47.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:47 smithi170 ceph-mon[33271]: pgmap v154: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:37:47.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:47 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/3613914158' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-17T08:37:47.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:47 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:47.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:47 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:47.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:47 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:37:47.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:47 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:47.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:47 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:47.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:47 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:37:47.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:47 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:37:47.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:47 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:47.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:47 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]: dispatch 2024-09-17T08:37:47.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:47 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]: dispatch 2024-09-17T08:37:47.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:47 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]': finished 2024-09-17T08:37:48.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:48 smithi118 ceph-mon[29491]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:37:48.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:48 smithi118 ceph-mon[29491]: Schedule rotate-key daemon osd.2 2024-09-17T08:37:48.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:48 smithi118 ceph-mon[29491]: Rotating authentication key for osd.2 2024-09-17T08:37:48.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:48 smithi118 ceph-mon[29491]: Reconfiguring daemon osd.2 on smithi118 2024-09-17T08:37:48.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:48 smithi118 ceph-mon[41418]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:37:48.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:48 smithi118 ceph-mon[41418]: Schedule rotate-key daemon osd.2 2024-09-17T08:37:48.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:48 smithi118 ceph-mon[41418]: Rotating authentication key for osd.2 2024-09-17T08:37:48.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:48 smithi118 ceph-mon[41418]: Reconfiguring daemon osd.2 on smithi118 2024-09-17T08:37:48.495 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:48 smithi170 ceph-mon[33271]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:37:48.495 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:48 smithi170 ceph-mon[33271]: Schedule rotate-key daemon osd.2 2024-09-17T08:37:48.495 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:48 smithi170 ceph-mon[33271]: Rotating authentication key for osd.2 2024-09-17T08:37:48.495 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:48 smithi170 ceph-mon[33271]: Reconfiguring daemon osd.2 on smithi118 2024-09-17T08:37:48.895 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:37:48 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:37:49.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:49 smithi118 ceph-mon[41418]: pgmap v155: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:49.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:49 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:49.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:49 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:49.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:49 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:49.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:49 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:49.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:49 smithi118 ceph-mon[29491]: pgmap v155: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:49.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:49 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:49.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:49 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:49.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:49 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:49.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:49 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:49.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:49 smithi170 ceph-mon[33271]: pgmap v155: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:49.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:49 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:49.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:49 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:49.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:49 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:49.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:49 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:37:50.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:50 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:37:50.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:50 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:37:50.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:50 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:37:51.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:51 smithi118 ceph-mon[29491]: pgmap v156: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:37:51.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:51 smithi118 ceph-mon[41418]: pgmap v156: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:37:51.576 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.2 2024-09-17T08:37:51.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:51 smithi170 ceph-mon[33271]: pgmap v156: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:37:51.909 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQDuPelm6EKBOhAAmJAejkSxjMM0DwWoNUdhlw== 2024-09-17T08:37:51.909 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQDuPelm6EKBOhAAmJAejkSxjMM0DwWoNUdhlw== == AQDuPelm6EKBOhAAmJAejkSxjMM0DwWoNUdhlw== ']' 2024-09-17T08:37:51.909 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:37:52.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:52 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/1620483806' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-17T08:37:52.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:52 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/1620483806' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-17T08:37:52.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:52 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/1620483806' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-17T08:37:53.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:53 smithi118 ceph-mon[41418]: pgmap v157: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:53.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:53 smithi118 ceph-mon[29491]: pgmap v157: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:53.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:53 smithi170 ceph-mon[33271]: pgmap v157: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:55.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:55 smithi118 ceph-mon[41418]: pgmap v158: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:55.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:55 smithi118 ceph-mon[29491]: pgmap v158: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:55.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:55 smithi170 ceph-mon[33271]: pgmap v158: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:56.911 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.2 2024-09-17T08:37:57.235 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQDuPelm6EKBOhAAmJAejkSxjMM0DwWoNUdhlw== 2024-09-17T08:37:57.235 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQDuPelm6EKBOhAAmJAejkSxjMM0DwWoNUdhlw== == AQDuPelm6EKBOhAAmJAejkSxjMM0DwWoNUdhlw== ']' 2024-09-17T08:37:57.235 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:37:57.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:57 smithi118 ceph-mon[29491]: pgmap v159: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:37:57.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:57 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3913361616' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-17T08:37:57.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:57 smithi118 ceph-mon[41418]: pgmap v159: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:37:57.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:57 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/3913361616' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-17T08:37:57.528 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:37:57 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:37:57] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:37:57.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:57 smithi170 ceph-mon[33271]: pgmap v159: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:37:57.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:57 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/3913361616' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-17T08:37:58.895 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:37:58 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:37:59.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:37:59 smithi118 ceph-mon[29491]: pgmap v160: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:59.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:37:59 smithi118 ceph-mon[41418]: pgmap v160: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:37:59.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:37:59 smithi170 ceph-mon[33271]: pgmap v160: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:00.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:00 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:38:00.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:00 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:38:00.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:00 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:38:00.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:00 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:38:00.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:00 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:38:00.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:00 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:38:01.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:01 smithi118 ceph-mon[29491]: pgmap v161: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:38:01.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:01 smithi118 ceph-mon[41418]: pgmap v161: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:38:01.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:01 smithi170 ceph-mon[33271]: pgmap v161: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:38:02.237 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.2 2024-09-17T08:38:02.563 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQDuPelm6EKBOhAAmJAejkSxjMM0DwWoNUdhlw== 2024-09-17T08:38:02.564 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQDuPelm6EKBOhAAmJAejkSxjMM0DwWoNUdhlw== == AQDuPelm6EKBOhAAmJAejkSxjMM0DwWoNUdhlw== ']' 2024-09-17T08:38:02.564 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:38:03.287 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:03 smithi118 ceph-mon[29491]: pgmap v162: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:03.287 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:03 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/2948486029' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-17T08:38:03.288 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:03 smithi118 ceph-mon[41418]: pgmap v162: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:03.288 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:03 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/2948486029' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-17T08:38:03.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:03 smithi170 ceph-mon[33271]: pgmap v162: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:03.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:03 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/2948486029' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-17T08:38:05.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:05 smithi118 ceph-mon[29491]: pgmap v163: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:05.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:05 smithi118 ceph-mon[41418]: pgmap v163: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:05.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:05 smithi170 ceph-mon[33271]: pgmap v163: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:07.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:07 smithi170 ceph-mon[33271]: pgmap v164: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:38:07.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:07 smithi118 ceph-mon[29491]: pgmap v164: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:38:07.527 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:38:07 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:38:07] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:38:07.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:07 smithi118 ceph-mon[41418]: pgmap v164: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:38:07.566 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.2 2024-09-17T08:38:07.889 INFO:teuthology.orchestra.run.smithi118.stdout:rotating key for osd.3 2024-09-17T08:38:07.890 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQDaP+lmnzgiNBAAJ2uNOOCNkrosZBaa65gRMA== 2024-09-17T08:38:07.890 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQDuPelm6EKBOhAAmJAejkSxjMM0DwWoNUdhlw== == AQDaP+lmnzgiNBAAJ2uNOOCNkrosZBaa65gRMA== ']' 2024-09-17T08:38:07.890 INFO:teuthology.orchestra.run.smithi118.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-17T08:38:07.890 INFO:teuthology.orchestra.run.smithi118.stderr:+ echo 'rotating key for osd.3' 2024-09-17T08:38:07.890 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.3 2024-09-17T08:38:08.204 INFO:teuthology.orchestra.run.smithi118.stderr:+ K=AQAIPulmAVH4JxAAtHEv8/zP1K97qdQW7jYtSw== 2024-09-17T08:38:08.205 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQAIPulmAVH4JxAAtHEv8/zP1K97qdQW7jYtSw== 2024-09-17T08:38:08.205 INFO:teuthology.orchestra.run.smithi118.stderr:+ ceph orch daemon rotate-key osd.3 2024-09-17T08:38:08.205 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:08 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3979449951' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-17T08:38:08.205 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:08 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/3979449951' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-17T08:38:08.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:08 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/3979449951' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-17T08:38:08.469 INFO:teuthology.orchestra.run.smithi118.stdout:Scheduled to rotate-key osd.3 on host 'smithi118' 2024-09-17T08:38:08.480 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQAIPulmAVH4JxAAtHEv8/zP1K97qdQW7jYtSw== == AQAIPulmAVH4JxAAtHEv8/zP1K97qdQW7jYtSw== ']' 2024-09-17T08:38:08.480 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:38:08.895 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:38:08 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:38:09.379 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:09 smithi118 ceph-mon[29491]: pgmap v165: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:09.379 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:09 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/2417305208' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-17T08:38:09.379 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:09 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:09.379 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:09 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:09.379 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:09 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:38:09.379 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:09 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:09.379 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:09 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:09.379 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:09 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:38:09.379 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:09 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:38:09.379 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:09 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:09.379 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:09 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]: dispatch 2024-09-17T08:38:09.380 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:09 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]: dispatch 2024-09-17T08:38:09.380 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:09 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]': finished 2024-09-17T08:38:09.380 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:09 smithi118 ceph-mon[41418]: pgmap v165: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:09.380 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:09 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/2417305208' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-17T08:38:09.380 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:09 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:09.380 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:09 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:09.380 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:09 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:38:09.380 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:09 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:09.380 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:09 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:09.381 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:09 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:38:09.381 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:09 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:38:09.381 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:09 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:09.381 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:09 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]: dispatch 2024-09-17T08:38:09.381 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:09 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]: dispatch 2024-09-17T08:38:09.381 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:09 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]': finished 2024-09-17T08:38:09.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:09 smithi170 ceph-mon[33271]: pgmap v165: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:09.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:09 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/2417305208' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-17T08:38:09.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:09 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:09.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:09 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:09.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:09 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:38:09.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:09 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:09.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:09 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:09.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:09 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:38:09.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:09 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:38:09.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:09 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:09.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:09 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]: dispatch 2024-09-17T08:38:09.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:09 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]: dispatch 2024-09-17T08:38:09.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:09 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]': finished 2024-09-17T08:38:10.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:10 smithi170 ceph-mon[33271]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:38:10.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:10 smithi170 ceph-mon[33271]: Schedule rotate-key daemon osd.3 2024-09-17T08:38:10.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:10 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:38:10.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:10 smithi170 ceph-mon[33271]: Rotating authentication key for osd.3 2024-09-17T08:38:10.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:10 smithi170 ceph-mon[33271]: Reconfiguring daemon osd.3 on smithi118 2024-09-17T08:38:10.502 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:10 smithi118 ceph-mon[29491]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:38:10.502 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:10 smithi118 ceph-mon[29491]: Schedule rotate-key daemon osd.3 2024-09-17T08:38:10.502 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:10 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:38:10.502 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:10 smithi118 ceph-mon[29491]: Rotating authentication key for osd.3 2024-09-17T08:38:10.502 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:10 smithi118 ceph-mon[29491]: Reconfiguring daemon osd.3 on smithi118 2024-09-17T08:38:10.502 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:10 smithi118 ceph-mon[41418]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:38:10.502 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:10 smithi118 ceph-mon[41418]: Schedule rotate-key daemon osd.3 2024-09-17T08:38:10.503 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:10 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:38:10.503 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:10 smithi118 ceph-mon[41418]: Rotating authentication key for osd.3 2024-09-17T08:38:10.503 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:10 smithi118 ceph-mon[41418]: Reconfiguring daemon osd.3 on smithi118 2024-09-17T08:38:11.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:11 smithi170 ceph-mon[33271]: pgmap v166: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:38:11.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:11 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:11.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:11 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:11.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:11 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:11.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:11 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:11.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:11 smithi118 ceph-mon[29491]: pgmap v166: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:38:11.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:11 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:11.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:11 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:11.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:11 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:11.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:11 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:11.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:11 smithi118 ceph-mon[41418]: pgmap v166: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:38:11.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:11 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:11.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:11 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:11.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:11 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:11.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:11 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:13.483 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.3 2024-09-17T08:38:13.483 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:13 smithi118 ceph-mon[29491]: pgmap v167: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:13.483 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:13 smithi118 ceph-mon[41418]: pgmap v167: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:13.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:13 smithi170 ceph-mon[33271]: pgmap v167: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:13.825 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQAIPulmAVH4JxAAtHEv8/zP1K97qdQW7jYtSw== 2024-09-17T08:38:13.825 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQAIPulmAVH4JxAAtHEv8/zP1K97qdQW7jYtSw== == AQAIPulmAVH4JxAAtHEv8/zP1K97qdQW7jYtSw== ']' 2024-09-17T08:38:13.825 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:38:14.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:14 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3024343268' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-17T08:38:14.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:14 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/3024343268' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-17T08:38:14.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:14 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/3024343268' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-17T08:38:15.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:15 smithi118 ceph-mon[29491]: pgmap v168: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:15.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:15 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:38:15.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:15 smithi118 ceph-mon[41418]: pgmap v168: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:15.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:15 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:38:15.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:15 smithi170 ceph-mon[33271]: pgmap v168: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:15.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:15 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:38:17.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:17 smithi118 ceph-mon[29491]: pgmap v169: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:38:17.527 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:38:17 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:38:17] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:38:17.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:17 smithi118 ceph-mon[41418]: pgmap v169: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:38:17.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:17 smithi170 ceph-mon[33271]: pgmap v169: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:38:18.828 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.3 2024-09-17T08:38:18.895 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:38:18 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:38:19.160 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQAIPulmAVH4JxAAtHEv8/zP1K97qdQW7jYtSw== 2024-09-17T08:38:19.160 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQAIPulmAVH4JxAAtHEv8/zP1K97qdQW7jYtSw== == AQAIPulmAVH4JxAAtHEv8/zP1K97qdQW7jYtSw== ']' 2024-09-17T08:38:19.160 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:38:19.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:19 smithi118 ceph-mon[29491]: pgmap v170: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:19.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:19 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/2857299457' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-17T08:38:19.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:19 smithi118 ceph-mon[41418]: pgmap v170: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:19.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:19 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/2857299457' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-17T08:38:19.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:19 smithi170 ceph-mon[33271]: pgmap v170: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:19.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:19 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/2857299457' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-17T08:38:20.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:20 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:38:20.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:20 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:38:20.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:20 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:38:21.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:21 smithi118 ceph-mon[29491]: pgmap v171: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:38:21.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:21 smithi118 ceph-mon[41418]: pgmap v171: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:38:21.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:21 smithi170 ceph-mon[33271]: pgmap v171: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:38:23.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:23 smithi118 ceph-mon[29491]: pgmap v172: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:23.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:23 smithi118 ceph-mon[41418]: pgmap v172: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:23.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:23 smithi170 ceph-mon[33271]: pgmap v172: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:24.162 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.3 2024-09-17T08:38:24.486 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQAIPulmAVH4JxAAtHEv8/zP1K97qdQW7jYtSw== 2024-09-17T08:38:24.487 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQAIPulmAVH4JxAAtHEv8/zP1K97qdQW7jYtSw== == AQAIPulmAVH4JxAAtHEv8/zP1K97qdQW7jYtSw== ']' 2024-09-17T08:38:24.487 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:38:25.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:25 smithi118 ceph-mon[29491]: pgmap v173: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:25.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:25 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/4235867740' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-17T08:38:25.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:25 smithi118 ceph-mon[41418]: pgmap v173: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:25.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:25 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/4235867740' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-17T08:38:25.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:25 smithi170 ceph-mon[33271]: pgmap v173: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:25.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:25 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/4235867740' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-17T08:38:27.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:27 smithi118 ceph-mon[29491]: pgmap v174: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:38:27.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:27 smithi118 ceph-mon[41418]: pgmap v174: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:38:27.528 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:38:27 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:38:27] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:38:27.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:27 smithi170 ceph-mon[33271]: pgmap v174: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:38:28.895 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:38:28 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:38:29.489 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.3 2024-09-17T08:38:29.489 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:29 smithi118 ceph-mon[29491]: pgmap v175: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:29.489 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:29 smithi118 ceph-mon[41418]: pgmap v175: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:29.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:29 smithi170 ceph-mon[33271]: pgmap v175: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:29.812 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQAIPulmAVH4JxAAtHEv8/zP1K97qdQW7jYtSw== 2024-09-17T08:38:29.813 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQAIPulmAVH4JxAAtHEv8/zP1K97qdQW7jYtSw== == AQAIPulmAVH4JxAAtHEv8/zP1K97qdQW7jYtSw== ']' 2024-09-17T08:38:29.813 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:38:30.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:30 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:38:30.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:30 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:38:30.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:30 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/81855268' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-17T08:38:30.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:30 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:38:30.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:30 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:38:30.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:30 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/81855268' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-17T08:38:30.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:30 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:38:30.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:30 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:38:30.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:30 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/81855268' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-17T08:38:31.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:31 smithi118 ceph-mon[29491]: pgmap v176: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:38:31.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:31 smithi118 ceph-mon[41418]: pgmap v176: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:38:31.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:31 smithi170 ceph-mon[33271]: pgmap v176: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:38:33.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:33 smithi118 ceph-mon[29491]: pgmap v177: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:33.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:33 smithi118 ceph-mon[41418]: pgmap v177: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:33.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:33 smithi170 ceph-mon[33271]: pgmap v177: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:34.815 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.3 2024-09-17T08:38:35.143 INFO:teuthology.orchestra.run.smithi118.stdout:rotating key for osd.4 2024-09-17T08:38:35.144 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQDwP+lmU1P2LhAAduj5SG/UMtx8DrXwEnpFyw== 2024-09-17T08:38:35.144 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQAIPulmAVH4JxAAtHEv8/zP1K97qdQW7jYtSw== == AQDwP+lmU1P2LhAAduj5SG/UMtx8DrXwEnpFyw== ']' 2024-09-17T08:38:35.144 INFO:teuthology.orchestra.run.smithi118.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-17T08:38:35.144 INFO:teuthology.orchestra.run.smithi118.stderr:+ echo 'rotating key for osd.4' 2024-09-17T08:38:35.144 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.4 2024-09-17T08:38:35.483 INFO:teuthology.orchestra.run.smithi118.stderr:+ K=AQAgPulmjJKXNBAApdP5f1oXxlzXtPxnB7A3Zg== 2024-09-17T08:38:35.483 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQAgPulmjJKXNBAApdP5f1oXxlzXtPxnB7A3Zg== 2024-09-17T08:38:35.483 INFO:teuthology.orchestra.run.smithi118.stderr:+ ceph orch daemon rotate-key osd.4 2024-09-17T08:38:35.484 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:35 smithi118 ceph-mon[29491]: pgmap v178: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:35.484 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:35 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/1263522000' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-17T08:38:35.484 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:35 smithi118 ceph-mon[41418]: pgmap v178: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:35.484 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:35 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/1263522000' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-17T08:38:35.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:35 smithi170 ceph-mon[33271]: pgmap v178: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:35.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:35 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/1263522000' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-17T08:38:35.769 INFO:teuthology.orchestra.run.smithi118.stdout:Scheduled to rotate-key osd.4 on host 'smithi170' 2024-09-17T08:38:35.780 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQAgPulmjJKXNBAApdP5f1oXxlzXtPxnB7A3Zg== == AQAgPulmjJKXNBAApdP5f1oXxlzXtPxnB7A3Zg== ']' 2024-09-17T08:38:35.780 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:38:36.487 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:36 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/1966604617' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-17T08:38:36.487 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:36 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:36.487 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:36 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:36.487 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:36 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:38:36.487 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:36 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:38:36.487 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:36 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:38:36.487 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:36 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:36.487 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:36 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]: dispatch 2024-09-17T08:38:36.487 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:36 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]: dispatch 2024-09-17T08:38:36.487 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:36 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]': finished 2024-09-17T08:38:36.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:36 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/1966604617' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-17T08:38:36.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:36 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:36.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:36 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:36.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:36 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:38:36.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:36 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:38:36.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:36 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:38:36.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:36 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:36.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:36 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]: dispatch 2024-09-17T08:38:36.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:36 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]: dispatch 2024-09-17T08:38:36.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:36 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]': finished 2024-09-17T08:38:36.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:36 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/1966604617' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-17T08:38:36.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:36 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:36.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:36 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:36.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:36 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:38:36.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:36 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:38:36.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:36 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:38:36.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:36 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:36.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:36 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]: dispatch 2024-09-17T08:38:36.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:36 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]: dispatch 2024-09-17T08:38:36.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:36 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]': finished 2024-09-17T08:38:37.527 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:38:37 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:38:37] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:38:37.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:37 smithi118 ceph-mon[29491]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:38:37.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:37 smithi118 ceph-mon[29491]: Schedule rotate-key daemon osd.4 2024-09-17T08:38:37.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:37 smithi118 ceph-mon[29491]: pgmap v179: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:38:37.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:37 smithi118 ceph-mon[29491]: Rotating authentication key for osd.4 2024-09-17T08:38:37.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:37 smithi118 ceph-mon[29491]: Reconfiguring daemon osd.4 on smithi170 2024-09-17T08:38:37.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:37 smithi118 ceph-mon[41418]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:38:37.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:37 smithi118 ceph-mon[41418]: Schedule rotate-key daemon osd.4 2024-09-17T08:38:37.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:37 smithi118 ceph-mon[41418]: pgmap v179: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:38:37.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:37 smithi118 ceph-mon[41418]: Rotating authentication key for osd.4 2024-09-17T08:38:37.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:37 smithi118 ceph-mon[41418]: Reconfiguring daemon osd.4 on smithi170 2024-09-17T08:38:37.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:37 smithi170 ceph-mon[33271]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:38:37.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:37 smithi170 ceph-mon[33271]: Schedule rotate-key daemon osd.4 2024-09-17T08:38:37.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:37 smithi170 ceph-mon[33271]: pgmap v179: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:38:37.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:37 smithi170 ceph-mon[33271]: Rotating authentication key for osd.4 2024-09-17T08:38:37.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:37 smithi170 ceph-mon[33271]: Reconfiguring daemon osd.4 on smithi170 2024-09-17T08:38:38.895 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:38:38 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:38:38.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:38 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:38.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:38 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:38.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:38 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:38.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:38 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:38.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:38 smithi170 ceph-mon[33271]: pgmap v180: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:39.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:38 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:39.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:38 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:39.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:38 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:39.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:38 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:39.027 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:38 smithi118 ceph-mon[29491]: pgmap v180: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:39.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:38 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:39.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:38 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:39.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:38 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:39.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:38 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:38:39.028 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:38 smithi118 ceph-mon[41418]: pgmap v180: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:40.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:40 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:38:40.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:40 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:38:40.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:40 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:38:40.782 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.4 2024-09-17T08:38:41.113 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQAgPulmjJKXNBAApdP5f1oXxlzXtPxnB7A3Zg== 2024-09-17T08:38:41.113 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQAgPulmjJKXNBAApdP5f1oXxlzXtPxnB7A3Zg== == AQAgPulmjJKXNBAApdP5f1oXxlzXtPxnB7A3Zg== ']' 2024-09-17T08:38:41.113 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:38:41.113 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:41 smithi118 ceph-mon[29491]: pgmap v181: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:38:41.114 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:41 smithi118 ceph-mon[41418]: pgmap v181: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:38:41.397 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:41 smithi170 ceph-mon[33271]: pgmap v181: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:38:42.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:42 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/2969331945' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-17T08:38:42.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:42 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/2969331945' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-17T08:38:42.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:42 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/2969331945' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-17T08:38:43.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:43 smithi170 ceph-mon[33271]: pgmap v182: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:43.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:43 smithi118 ceph-mon[29491]: pgmap v182: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:43.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:43 smithi118 ceph-mon[41418]: pgmap v182: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:45.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:45 smithi118 ceph-mon[29491]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:45.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:45 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:38:45.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:45 smithi118 ceph-mon[41418]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:45.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:45 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:38:45.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:45 smithi170 ceph-mon[33271]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:45.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:45 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:38:46.115 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.4 2024-09-17T08:38:46.444 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQAgPulmjJKXNBAApdP5f1oXxlzXtPxnB7A3Zg== 2024-09-17T08:38:46.444 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQAgPulmjJKXNBAApdP5f1oXxlzXtPxnB7A3Zg== == AQAgPulmjJKXNBAApdP5f1oXxlzXtPxnB7A3Zg== ']' 2024-09-17T08:38:46.444 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:38:47.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:47 smithi118 ceph-mon[29491]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:38:47.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:47 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/2721079725' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-17T08:38:47.528 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:38:47 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:38:47] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:38:47.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:47 smithi118 ceph-mon[41418]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:38:47.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:47 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/2721079725' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-17T08:38:47.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:47 smithi170 ceph-mon[33271]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:38:47.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:47 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/2721079725' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-17T08:38:48.895 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:38:48 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:38:49.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:49 smithi118 ceph-mon[29491]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:49.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:49 smithi118 ceph-mon[41418]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:49.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:49 smithi170 ceph-mon[33271]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:50.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:50 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:38:50.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:50 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:38:50.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:50 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:38:51.445 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.4 2024-09-17T08:38:51.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:51 smithi118 ceph-mon[29491]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:38:51.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:51 smithi118 ceph-mon[41418]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:38:51.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:51 smithi170 ceph-mon[33271]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:38:51.771 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQAgPulmjJKXNBAApdP5f1oXxlzXtPxnB7A3Zg== 2024-09-17T08:38:51.771 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQAgPulmjJKXNBAApdP5f1oXxlzXtPxnB7A3Zg== == AQAgPulmjJKXNBAApdP5f1oXxlzXtPxnB7A3Zg== ']' 2024-09-17T08:38:51.771 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:38:52.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:52 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/385855649' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-17T08:38:52.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:52 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/385855649' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-17T08:38:52.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:52 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/385855649' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-17T08:38:53.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:53 smithi118 ceph-mon[29491]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:53.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:53 smithi118 ceph-mon[41418]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:53.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:53 smithi170 ceph-mon[33271]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:55.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:55 smithi118 ceph-mon[29491]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:55.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:55 smithi118 ceph-mon[41418]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:55.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:55 smithi170 ceph-mon[33271]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:56.773 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.4 2024-09-17T08:38:57.095 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQAgPulmjJKXNBAApdP5f1oXxlzXtPxnB7A3Zg== 2024-09-17T08:38:57.095 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQAgPulmjJKXNBAApdP5f1oXxlzXtPxnB7A3Zg== == AQAgPulmjJKXNBAApdP5f1oXxlzXtPxnB7A3Zg== ']' 2024-09-17T08:38:57.095 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:38:57.527 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:38:57 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:38:57] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:38:57.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:57 smithi118 ceph-mon[29491]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:38:57.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:57 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/2015814784' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-17T08:38:57.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:57 smithi118 ceph-mon[41418]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:38:57.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:57 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/2015814784' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-17T08:38:57.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:57 smithi170 ceph-mon[33271]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:38:57.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:57 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/2015814784' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-17T08:38:58.895 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:38:58 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:38:59.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:58 smithi118 ceph-mon[29491]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:59.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:58 smithi118 ceph-mon[41418]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:38:59.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:58 smithi170 ceph-mon[33271]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:00.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:59 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:39:00.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:38:59 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:39:00.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:59 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:39:00.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:38:59 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:39:00.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:59 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:39:00.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:38:59 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:39:01.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:00 smithi118 ceph-mon[29491]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:39:01.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:00 smithi118 ceph-mon[41418]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:39:01.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:00 smithi170 ceph-mon[33271]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:39:02.097 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.4 2024-09-17T08:39:02.426 INFO:teuthology.orchestra.run.smithi118.stdout:rotating key for osd.5 2024-09-17T08:39:02.426 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQAMQOlmK4FkChAAnAexr+3F5i3XBgdvrJW9MQ== 2024-09-17T08:39:02.426 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQAgPulmjJKXNBAApdP5f1oXxlzXtPxnB7A3Zg== == AQAMQOlmK4FkChAAnAexr+3F5i3XBgdvrJW9MQ== ']' 2024-09-17T08:39:02.426 INFO:teuthology.orchestra.run.smithi118.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-17T08:39:02.426 INFO:teuthology.orchestra.run.smithi118.stderr:+ echo 'rotating key for osd.5' 2024-09-17T08:39:02.426 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.5 2024-09-17T08:39:02.741 INFO:teuthology.orchestra.run.smithi118.stderr:+ K=AQA4PulmTzpbChAADA9PX757o26r5wYX3NC2rQ== 2024-09-17T08:39:02.741 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQA4PulmTzpbChAADA9PX757o26r5wYX3NC2rQ== 2024-09-17T08:39:02.741 INFO:teuthology.orchestra.run.smithi118.stderr:+ ceph orch daemon rotate-key osd.5 2024-09-17T08:39:03.007 INFO:teuthology.orchestra.run.smithi118.stdout:Scheduled to rotate-key osd.5 on host 'smithi170' 2024-09-17T08:39:03.020 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQA4PulmTzpbChAADA9PX757o26r5wYX3NC2rQ== == AQA4PulmTzpbChAADA9PX757o26r5wYX3NC2rQ== ']' 2024-09-17T08:39:03.021 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:39:03.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:03 smithi118 ceph-mon[29491]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:03.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:03 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3920930506' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-17T08:39:03.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:03 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3642893070' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-17T08:39:03.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:03 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:03.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:03 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:03.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:03 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:39:03.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:03 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:39:03.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:03 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:39:03.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:03 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:03.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:03 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]: dispatch 2024-09-17T08:39:03.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:03 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]: dispatch 2024-09-17T08:39:03.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:03 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]': finished 2024-09-17T08:39:03.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:03 smithi118 ceph-mon[41418]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:03.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:03 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/3920930506' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-17T08:39:03.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:03 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/3642893070' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-17T08:39:03.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:03 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:03.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:03 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:03.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:03 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:39:03.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:03 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:39:03.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:03 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:39:03.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:03 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:03.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:03 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]: dispatch 2024-09-17T08:39:03.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:03 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]: dispatch 2024-09-17T08:39:03.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:03 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]': finished 2024-09-17T08:39:03.539 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:03 smithi170 ceph-mon[33271]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:03.539 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:03 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/3920930506' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-17T08:39:03.539 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:03 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/3642893070' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-17T08:39:03.539 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:03 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:03.539 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:03 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:03.539 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:03 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:39:03.539 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:03 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:39:03.539 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:03 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:39:03.539 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:03 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:03.539 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:03 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]: dispatch 2024-09-17T08:39:03.540 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:03 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]: dispatch 2024-09-17T08:39:03.540 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:03 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]': finished 2024-09-17T08:39:04.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:04 smithi118 ceph-mon[29491]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.5", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:39:04.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:04 smithi118 ceph-mon[29491]: Schedule rotate-key daemon osd.5 2024-09-17T08:39:04.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:04 smithi118 ceph-mon[29491]: Rotating authentication key for osd.5 2024-09-17T08:39:04.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:04 smithi118 ceph-mon[29491]: Reconfiguring daemon osd.5 on smithi170 2024-09-17T08:39:04.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:04 smithi118 ceph-mon[41418]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.5", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:39:04.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:04 smithi118 ceph-mon[41418]: Schedule rotate-key daemon osd.5 2024-09-17T08:39:04.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:04 smithi118 ceph-mon[41418]: Rotating authentication key for osd.5 2024-09-17T08:39:04.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:04 smithi118 ceph-mon[41418]: Reconfiguring daemon osd.5 on smithi170 2024-09-17T08:39:04.534 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:04 smithi170 ceph-mon[33271]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.5", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:39:04.534 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:04 smithi170 ceph-mon[33271]: Schedule rotate-key daemon osd.5 2024-09-17T08:39:04.535 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:04 smithi170 ceph-mon[33271]: Rotating authentication key for osd.5 2024-09-17T08:39:04.535 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:04 smithi170 ceph-mon[33271]: Reconfiguring daemon osd.5 on smithi170 2024-09-17T08:39:05.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:05 smithi118 ceph-mon[29491]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:05.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:05 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:05.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:05 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:05.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:05 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:05.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:05 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:05.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:05 smithi118 ceph-mon[41418]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:05.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:05 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:05.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:05 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:05.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:05 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:05.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:05 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:05.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:05 smithi170 ceph-mon[33271]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:05.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:05 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:05.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:05 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:05.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:05 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:05.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:05 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:07.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:07 smithi118 ceph-mon[29491]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:39:07.527 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:39:07 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:39:07] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:39:07.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:07 smithi118 ceph-mon[41418]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:39:07.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:07 smithi170 ceph-mon[33271]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:39:08.023 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.5 2024-09-17T08:39:08.350 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQA4PulmTzpbChAADA9PX757o26r5wYX3NC2rQ== 2024-09-17T08:39:08.350 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQA4PulmTzpbChAADA9PX757o26r5wYX3NC2rQ== == AQA4PulmTzpbChAADA9PX757o26r5wYX3NC2rQ== ']' 2024-09-17T08:39:08.350 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:39:08.895 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:39:08 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:39:09.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:09 smithi118 ceph-mon[29491]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:09.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:09 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/486789584' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-17T08:39:09.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:09 smithi118 ceph-mon[41418]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:09.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:09 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/486789584' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-17T08:39:09.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:09 smithi170 ceph-mon[33271]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:09.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:09 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/486789584' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-17T08:39:10.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:10 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:39:10.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:10 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:39:10.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:10 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:39:11.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:11 smithi118 ceph-mon[29491]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:39:11.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:11 smithi118 ceph-mon[41418]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:39:11.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:11 smithi170 ceph-mon[33271]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:39:13.352 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.5 2024-09-17T08:39:13.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:13 smithi118 ceph-mon[29491]: pgmap v197: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:13.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:13 smithi118 ceph-mon[41418]: pgmap v197: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:13.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:13 smithi170 ceph-mon[33271]: pgmap v197: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:13.677 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQA4PulmTzpbChAADA9PX757o26r5wYX3NC2rQ== 2024-09-17T08:39:13.677 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQA4PulmTzpbChAADA9PX757o26r5wYX3NC2rQ== == AQA4PulmTzpbChAADA9PX757o26r5wYX3NC2rQ== ']' 2024-09-17T08:39:13.677 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:39:14.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:14 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/2556562661' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-17T08:39:14.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:14 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/2556562661' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-17T08:39:14.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:14 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/2556562661' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-17T08:39:15.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:15 smithi118 ceph-mon[29491]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:15.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:15 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:39:15.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:15 smithi118 ceph-mon[41418]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:15.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:15 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:39:15.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:15 smithi170 ceph-mon[33271]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:15.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:15 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:39:17.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:17 smithi118 ceph-mon[29491]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:39:17.527 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:39:17 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:39:17] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:39:17.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:17 smithi118 ceph-mon[41418]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:39:17.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:17 smithi170 ceph-mon[33271]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:39:18.678 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.5 2024-09-17T08:39:18.895 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:39:18 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:39:19.005 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQAnQOlmMWqQBRAAUjGfrRyq9dHVvFo8MPmOkw== 2024-09-17T08:39:19.005 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQA4PulmTzpbChAADA9PX757o26r5wYX3NC2rQ== == AQAnQOlmMWqQBRAAUjGfrRyq9dHVvFo8MPmOkw== ']' 2024-09-17T08:39:19.005 INFO:teuthology.orchestra.run.smithi118.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-17T08:39:19.005 INFO:teuthology.orchestra.run.smithi118.stderr:+ echo 'rotating key for osd.6' 2024-09-17T08:39:19.006 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:18 smithi118 ceph-mon[29491]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:19.006 INFO:teuthology.orchestra.run.smithi118.stdout:rotating key for osd.6 2024-09-17T08:39:19.006 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:18 smithi118 ceph-mon[41418]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:19.007 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.6 2024-09-17T08:39:19.318 INFO:teuthology.orchestra.run.smithi118.stderr:+ K=AQBPPulm8c0/ORAAYTgzKMPNoyAOvr0T7bc2sQ== 2024-09-17T08:39:19.318 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQBPPulm8c0/ORAAYTgzKMPNoyAOvr0T7bc2sQ== 2024-09-17T08:39:19.318 INFO:teuthology.orchestra.run.smithi118.stderr:+ ceph orch daemon rotate-key osd.6 2024-09-17T08:39:19.397 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:18 smithi170 ceph-mon[33271]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:19.582 INFO:teuthology.orchestra.run.smithi118.stdout:Scheduled to rotate-key osd.6 on host 'smithi170' 2024-09-17T08:39:19.595 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQBPPulm8c0/ORAAYTgzKMPNoyAOvr0T7bc2sQ== == AQBPPulm8c0/ORAAYTgzKMPNoyAOvr0T7bc2sQ== ']' 2024-09-17T08:39:19.595 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:39:20.271 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:19 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:39:20.271 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:19 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/1700015663' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-17T08:39:20.271 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:19 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/1617041738' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-17T08:39:20.271 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:19 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:20.271 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:19 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:20.271 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:19 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:39:20.271 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:19 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:39:20.271 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:19 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:39:20.272 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:19 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:20.272 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:19 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]: dispatch 2024-09-17T08:39:20.272 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:19 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]: dispatch 2024-09-17T08:39:20.272 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:19 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]': finished 2024-09-17T08:39:20.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:19 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:39:20.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:19 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/1700015663' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-17T08:39:20.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:19 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/1617041738' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-17T08:39:20.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:19 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:20.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:19 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:20.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:19 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:39:20.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:19 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:39:20.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:19 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:39:20.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:19 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:20.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:19 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]: dispatch 2024-09-17T08:39:20.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:19 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]: dispatch 2024-09-17T08:39:20.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:19 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]': finished 2024-09-17T08:39:20.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:19 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:39:20.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:19 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/1700015663' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-17T08:39:20.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:19 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/1617041738' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-17T08:39:20.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:19 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:20.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:19 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:20.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:19 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:39:20.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:19 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:39:20.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:19 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:39:20.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:19 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:20.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:19 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]: dispatch 2024-09-17T08:39:20.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:19 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]: dispatch 2024-09-17T08:39:20.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:19 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]': finished 2024-09-17T08:39:21.033 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:20 smithi170 ceph-mon[33271]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.6", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:39:21.036 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:20 smithi170 ceph-mon[33271]: Schedule rotate-key daemon osd.6 2024-09-17T08:39:21.036 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:20 smithi170 ceph-mon[33271]: Rotating authentication key for osd.6 2024-09-17T08:39:21.036 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:20 smithi170 ceph-mon[33271]: Reconfiguring daemon osd.6 on smithi170 2024-09-17T08:39:21.037 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:20 smithi170 ceph-mon[33271]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:39:21.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:20 smithi118 ceph-mon[41418]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.6", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:39:21.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:20 smithi118 ceph-mon[41418]: Schedule rotate-key daemon osd.6 2024-09-17T08:39:21.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:20 smithi118 ceph-mon[41418]: Rotating authentication key for osd.6 2024-09-17T08:39:21.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:20 smithi118 ceph-mon[41418]: Reconfiguring daemon osd.6 on smithi170 2024-09-17T08:39:21.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:20 smithi118 ceph-mon[41418]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:39:21.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:20 smithi118 ceph-mon[29491]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.6", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:39:21.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:20 smithi118 ceph-mon[29491]: Schedule rotate-key daemon osd.6 2024-09-17T08:39:21.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:20 smithi118 ceph-mon[29491]: Rotating authentication key for osd.6 2024-09-17T08:39:21.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:20 smithi118 ceph-mon[29491]: Reconfiguring daemon osd.6 on smithi170 2024-09-17T08:39:21.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:20 smithi118 ceph-mon[29491]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:39:22.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:22 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:22.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:22 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:22.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:22 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:22.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:22 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:22.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:22 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:22.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:22 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:22.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:22 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:22.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:22 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:22.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:22 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:22.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:22 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:22.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:22 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:22.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:22 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:23.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:23 smithi170 ceph-mon[33271]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:23.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:23 smithi118 ceph-mon[29491]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:23.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:23 smithi118 ceph-mon[41418]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:24.597 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.6 2024-09-17T08:39:24.927 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQBPPulm8c0/ORAAYTgzKMPNoyAOvr0T7bc2sQ== 2024-09-17T08:39:24.927 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQBPPulm8c0/ORAAYTgzKMPNoyAOvr0T7bc2sQ== == AQBPPulm8c0/ORAAYTgzKMPNoyAOvr0T7bc2sQ== ']' 2024-09-17T08:39:24.927 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:39:25.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:25 smithi118 ceph-mon[29491]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:25.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:25 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/2846212290' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-17T08:39:25.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:25 smithi118 ceph-mon[41418]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:25.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:25 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/2846212290' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-17T08:39:25.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:25 smithi170 ceph-mon[33271]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:25.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:25 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/2846212290' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-17T08:39:27.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:27 smithi118 ceph-mon[29491]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:39:27.528 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:39:27 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:39:27] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:39:27.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:27 smithi118 ceph-mon[41418]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:39:27.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:27 smithi170 ceph-mon[33271]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:39:28.895 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:39:28 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:39:29.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:29 smithi118 ceph-mon[29491]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:29.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:29 smithi118 ceph-mon[41418]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:29.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:29 smithi170 ceph-mon[33271]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:29.929 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.6 2024-09-17T08:39:30.257 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQBPPulm8c0/ORAAYTgzKMPNoyAOvr0T7bc2sQ== 2024-09-17T08:39:30.257 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQBPPulm8c0/ORAAYTgzKMPNoyAOvr0T7bc2sQ== == AQBPPulm8c0/ORAAYTgzKMPNoyAOvr0T7bc2sQ== ']' 2024-09-17T08:39:30.257 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:39:30.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:30 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:39:30.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:30 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:39:30.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:30 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:39:30.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:30 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:39:30.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:30 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:39:30.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:30 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:39:31.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:31 smithi118 ceph-mon[29491]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:39:31.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:31 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/848211608' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-17T08:39:31.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:31 smithi118 ceph-mon[41418]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:39:31.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:31 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/848211608' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-17T08:39:31.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:31 smithi170 ceph-mon[33271]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:39:31.648 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:31 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/848211608' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-17T08:39:33.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:33 smithi118 ceph-mon[29491]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:33.529 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:33 smithi118 ceph-mon[41418]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:33.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:33 smithi170 ceph-mon[33271]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:35.258 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.6 2024-09-17T08:39:35.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:35 smithi118 ceph-mon[29491]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:35.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:35 smithi118 ceph-mon[41418]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:35.583 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQBPPulm8c0/ORAAYTgzKMPNoyAOvr0T7bc2sQ== 2024-09-17T08:39:35.583 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQBPPulm8c0/ORAAYTgzKMPNoyAOvr0T7bc2sQ== == AQBPPulm8c0/ORAAYTgzKMPNoyAOvr0T7bc2sQ== ']' 2024-09-17T08:39:35.583 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:39:35.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:35 smithi170 ceph-mon[33271]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:36.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:36 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/2800616872' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-17T08:39:36.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:36 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/2800616872' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-17T08:39:36.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:36 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/2800616872' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-17T08:39:37.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:37 smithi118 ceph-mon[29491]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:39:37.527 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:39:37 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:39:37] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:39:37.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:37 smithi118 ceph-mon[41418]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:39:37.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:37 smithi170 ceph-mon[33271]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:39:38.895 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:39:38 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:39:39.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:39 smithi118 ceph-mon[29491]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:39.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:39 smithi118 ceph-mon[41418]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:39.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:39 smithi170 ceph-mon[33271]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:40.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:40 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:39:40.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:40 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:39:40.585 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.6 2024-09-17T08:39:40.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:40 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:39:40.908 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQBPPulm8c0/ORAAYTgzKMPNoyAOvr0T7bc2sQ== 2024-09-17T08:39:40.908 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQBPPulm8c0/ORAAYTgzKMPNoyAOvr0T7bc2sQ== == AQBPPulm8c0/ORAAYTgzKMPNoyAOvr0T7bc2sQ== ']' 2024-09-17T08:39:40.908 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:39:41.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:41 smithi118 ceph-mon[29491]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:39:41.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:41 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3455201485' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-17T08:39:41.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:41 smithi118 ceph-mon[41418]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:39:41.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:41 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/3455201485' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-17T08:39:41.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:41 smithi170 ceph-mon[33271]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:39:41.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:41 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/3455201485' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-17T08:39:43.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:43 smithi118 ceph-mon[29491]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:43.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:43 smithi118 ceph-mon[41418]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:43.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:43 smithi170 ceph-mon[33271]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:45.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:45 smithi118 ceph-mon[29491]: pgmap v213: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:45.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:45 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:39:45.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:45 smithi118 ceph-mon[41418]: pgmap v213: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:45.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:45 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:39:45.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:45 smithi170 ceph-mon[33271]: pgmap v213: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:45.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:45 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:39:45.910 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.6 2024-09-17T08:39:46.241 INFO:teuthology.orchestra.run.smithi118.stdout:rotating key for osd.7 2024-09-17T08:39:46.242 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQA3QOlm/ZG5JxAAF+7zRbaZowqK7HlIuBkL7Q== 2024-09-17T08:39:46.242 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQBPPulm8c0/ORAAYTgzKMPNoyAOvr0T7bc2sQ== == AQA3QOlm/ZG5JxAAF+7zRbaZowqK7HlIuBkL7Q== ']' 2024-09-17T08:39:46.242 INFO:teuthology.orchestra.run.smithi118.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-17T08:39:46.242 INFO:teuthology.orchestra.run.smithi118.stderr:+ echo 'rotating key for osd.7' 2024-09-17T08:39:46.242 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.7 2024-09-17T08:39:46.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:46 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/1387560114' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-17T08:39:46.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:46 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/1387560114' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-17T08:39:46.562 INFO:teuthology.orchestra.run.smithi118.stderr:+ K=AQBnPulm5QExNhAAE5RzG2bCEHD3Z1RE3Fc0QQ== 2024-09-17T08:39:46.562 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQBnPulm5QExNhAAE5RzG2bCEHD3Z1RE3Fc0QQ== 2024-09-17T08:39:46.562 INFO:teuthology.orchestra.run.smithi118.stderr:+ ceph orch daemon rotate-key osd.7 2024-09-17T08:39:46.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:46 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/1387560114' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-17T08:39:46.829 INFO:teuthology.orchestra.run.smithi118.stdout:Scheduled to rotate-key osd.7 on host 'smithi170' 2024-09-17T08:39:46.841 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQBnPulm5QExNhAAE5RzG2bCEHD3Z1RE3Fc0QQ== == AQBnPulm5QExNhAAE5RzG2bCEHD3Z1RE3Fc0QQ== ']' 2024-09-17T08:39:46.841 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:39:47.527 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:39:47 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:39:47] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:39:47.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:47 smithi118 ceph-mon[41418]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:39:47.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:47 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/1920753991' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-17T08:39:47.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:47 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:47.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:47 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:47.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:47 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:39:47.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:47 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:39:47.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:47 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:39:47.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:47 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:47.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:47 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]: dispatch 2024-09-17T08:39:47.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:47 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]: dispatch 2024-09-17T08:39:47.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:47 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]': finished 2024-09-17T08:39:47.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:47 smithi118 ceph-mon[29491]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:39:47.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:47 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/1920753991' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-17T08:39:47.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:47 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:47.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:47 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:47.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:47 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:39:47.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:47 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:39:47.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:47 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:39:47.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:47 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:47.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:47 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]: dispatch 2024-09-17T08:39:47.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:47 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]: dispatch 2024-09-17T08:39:47.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:47 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]': finished 2024-09-17T08:39:47.541 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:47 smithi170 ceph-mon[33271]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:39:47.541 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:47 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/1920753991' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-17T08:39:47.541 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:47 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:47.541 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:47 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:47.541 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:47 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:39:47.541 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:47 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:39:47.541 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:47 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:39:47.542 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:47 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:47.542 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:47 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]: dispatch 2024-09-17T08:39:47.542 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:47 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]: dispatch 2024-09-17T08:39:47.542 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:47 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]': finished 2024-09-17T08:39:48.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:48 smithi118 ceph-mon[29491]: from='client.25148 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.7", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:39:48.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:48 smithi118 ceph-mon[29491]: Schedule rotate-key daemon osd.7 2024-09-17T08:39:48.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:48 smithi118 ceph-mon[29491]: Rotating authentication key for osd.7 2024-09-17T08:39:48.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:48 smithi118 ceph-mon[29491]: Reconfiguring daemon osd.7 on smithi170 2024-09-17T08:39:48.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:48 smithi118 ceph-mon[41418]: from='client.25148 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.7", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:39:48.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:48 smithi118 ceph-mon[41418]: Schedule rotate-key daemon osd.7 2024-09-17T08:39:48.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:48 smithi118 ceph-mon[41418]: Rotating authentication key for osd.7 2024-09-17T08:39:48.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:48 smithi118 ceph-mon[41418]: Reconfiguring daemon osd.7 on smithi170 2024-09-17T08:39:48.529 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:48 smithi170 ceph-mon[33271]: from='client.25148 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.7", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:39:48.529 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:48 smithi170 ceph-mon[33271]: Schedule rotate-key daemon osd.7 2024-09-17T08:39:48.529 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:48 smithi170 ceph-mon[33271]: Rotating authentication key for osd.7 2024-09-17T08:39:48.529 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:48 smithi170 ceph-mon[33271]: Reconfiguring daemon osd.7 on smithi170 2024-09-17T08:39:48.895 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:39:48 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:39:49.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:49 smithi118 ceph-mon[29491]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:49.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:49 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:49.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:49 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:49.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:49 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:49.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:49 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:49.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:49 smithi118 ceph-mon[41418]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:49.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:49 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:49.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:49 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:49.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:49 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:49.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:49 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:49.620 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:49 smithi170 ceph-mon[33271]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:49.620 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:49 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:49.620 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:49 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:49.620 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:49 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:49.620 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:49 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:39:50.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:50 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:39:50.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:50 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:39:50.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:50 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:39:51.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:51 smithi118 ceph-mon[29491]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:39:51.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:51 smithi118 ceph-mon[41418]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:39:51.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:51 smithi170 ceph-mon[33271]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:39:51.842 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.7 2024-09-17T08:39:52.175 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQBnPulm5QExNhAAE5RzG2bCEHD3Z1RE3Fc0QQ== 2024-09-17T08:39:52.175 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQBnPulm5QExNhAAE5RzG2bCEHD3Z1RE3Fc0QQ== == AQBnPulm5QExNhAAE5RzG2bCEHD3Z1RE3Fc0QQ== ']' 2024-09-17T08:39:52.176 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:39:52.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:52 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/95207273' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-17T08:39:52.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:52 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/95207273' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-17T08:39:52.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:52 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/95207273' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-17T08:39:53.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:53 smithi118 ceph-mon[29491]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:53.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:53 smithi118 ceph-mon[41418]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:53.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:53 smithi170 ceph-mon[33271]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:55.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:55 smithi118 ceph-mon[29491]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:55.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:55 smithi118 ceph-mon[41418]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:55.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:55 smithi170 ceph-mon[33271]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:57.177 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.7 2024-09-17T08:39:57.502 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQBnPulm5QExNhAAE5RzG2bCEHD3Z1RE3Fc0QQ== 2024-09-17T08:39:57.503 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQBnPulm5QExNhAAE5RzG2bCEHD3Z1RE3Fc0QQ== == AQBnPulm5QExNhAAE5RzG2bCEHD3Z1RE3Fc0QQ== ']' 2024-09-17T08:39:57.503 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:39:57.503 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:57 smithi118 ceph-mon[29491]: pgmap v219: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:39:57.503 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:39:57 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:39:57] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:39:57.504 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:57 smithi118 ceph-mon[41418]: pgmap v219: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:39:57.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:57 smithi170 ceph-mon[33271]: pgmap v219: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:39:58.614 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:58 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/3656872658' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-17T08:39:58.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:58 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3656872658' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-17T08:39:58.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:58 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/3656872658' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-17T08:39:58.895 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:39:58 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:39:59.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:39:59 smithi170 ceph-mon[33271]: pgmap v220: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:59.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:39:59 smithi118 ceph-mon[29491]: pgmap v220: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:39:59.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:39:59 smithi118 ceph-mon[41418]: pgmap v220: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:40:00.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:00 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:40:00.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:00 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:40:00.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:00 smithi170 ceph-mon[33271]: overall HEALTH_OK 2024-09-17T08:40:00.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:00 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:40:00.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:00 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:40:00.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:00 smithi118 ceph-mon[29491]: overall HEALTH_OK 2024-09-17T08:40:00.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:00 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:40:00.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:00 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:40:00.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:00 smithi118 ceph-mon[41418]: overall HEALTH_OK 2024-09-17T08:40:01.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:01 smithi170 ceph-mon[33271]: pgmap v221: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:40:01.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:01 smithi118 ceph-mon[29491]: pgmap v221: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:40:01.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:01 smithi118 ceph-mon[41418]: pgmap v221: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:40:02.504 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.7 2024-09-17T08:40:02.832 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQBnPulm5QExNhAAE5RzG2bCEHD3Z1RE3Fc0QQ== 2024-09-17T08:40:02.833 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQBnPulm5QExNhAAE5RzG2bCEHD3Z1RE3Fc0QQ== == AQBnPulm5QExNhAAE5RzG2bCEHD3Z1RE3Fc0QQ== ']' 2024-09-17T08:40:02.833 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:40:03.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:03 smithi170 ceph-mon[33271]: pgmap v222: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:40:03.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:03 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/86393084' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-17T08:40:03.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:03 smithi118 ceph-mon[29491]: pgmap v222: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:40:03.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:03 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/86393084' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-17T08:40:03.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:03 smithi118 ceph-mon[41418]: pgmap v222: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:40:03.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:03 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/86393084' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-17T08:40:05.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:05 smithi170 ceph-mon[33271]: pgmap v223: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:40:05.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:05 smithi118 ceph-mon[29491]: pgmap v223: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:40:05.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:05 smithi118 ceph-mon[41418]: pgmap v223: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:40:07.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:07 smithi118 ceph-mon[29491]: pgmap v224: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:40:07.527 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:07 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:40:07] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:40:07.528 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:07 smithi118 ceph-mon[41418]: pgmap v224: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:40:07.645 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:07 smithi170 ceph-mon[33271]: pgmap v224: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:40:07.834 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key osd.7 2024-09-17T08:40:08.160 INFO:teuthology.orchestra.run.smithi118.stdout:rotating key for mgr.y 2024-09-17T08:40:08.161 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQBTQOlmnMinDRAA8hvyLCCnFVQcb5+oGwO5MQ== 2024-09-17T08:40:08.161 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQBnPulm5QExNhAAE5RzG2bCEHD3Z1RE3Fc0QQ== == AQBTQOlmnMinDRAA8hvyLCCnFVQcb5+oGwO5MQ== ']' 2024-09-17T08:40:08.161 INFO:teuthology.orchestra.run.smithi118.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-17T08:40:08.161 INFO:teuthology.orchestra.run.smithi118.stderr:+ echo 'rotating key for mgr.y' 2024-09-17T08:40:08.161 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key mgr.y 2024-09-17T08:40:08.472 INFO:teuthology.orchestra.run.smithi118.stderr:+ K=AQA+Pelmu2hOOxAAXh7oN9t3ukF9M4Mmr9NejQ== 2024-09-17T08:40:08.472 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQA+Pelmu2hOOxAAXh7oN9t3ukF9M4Mmr9NejQ== 2024-09-17T08:40:08.472 INFO:teuthology.orchestra.run.smithi118.stderr:+ ceph orch daemon rotate-key mgr.y 2024-09-17T08:40:08.737 INFO:teuthology.orchestra.run.smithi118.stdout:Scheduled to rotate-key mgr.y on host 'smithi118' 2024-09-17T08:40:08.749 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQA+Pelmu2hOOxAAXh7oN9t3ukF9M4Mmr9NejQ== == AQA+Pelmu2hOOxAAXh7oN9t3ukF9M4Mmr9NejQ== ']' 2024-09-17T08:40:08.749 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:40:08.895 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:40:08 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:40:09.023 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:08 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/4192947577' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-17T08:40:09.023 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:08 smithi118 ceph-mon[29491]: pgmap v225: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:40:09.023 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:08 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/487044577' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-17T08:40:09.023 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:08 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:40:09.023 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:08 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:40:09.023 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:08 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:40:09.023 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:08 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:40:09.023 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:08 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:40:09.023 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:08 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:40:09.023 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:08 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-17T08:40:09.024 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:08 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-17T08:40:09.024 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:08 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-17T08:40:09.024 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:08 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:40:09.024 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:08 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:40:09.024 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:08 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T08:40:09.024 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:08 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:40:09.024 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:08 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/4192947577' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-17T08:40:09.024 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:08 smithi118 ceph-mon[41418]: pgmap v225: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:40:09.024 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:08 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/487044577' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-17T08:40:09.025 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:08 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:40:09.025 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:08 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:40:09.025 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:08 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:40:09.025 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:08 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:40:09.025 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:08 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:40:09.025 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:08 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:40:09.025 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:08 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-17T08:40:09.025 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:08 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-17T08:40:09.025 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:08 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-17T08:40:09.025 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:08 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:40:09.025 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:08 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:40:09.025 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:08 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T08:40:09.025 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:08 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:40:09.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:08 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/4192947577' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-17T08:40:09.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:08 smithi170 ceph-mon[33271]: pgmap v225: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:40:09.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:08 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/487044577' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-17T08:40:09.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:08 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:40:09.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:08 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:40:09.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:08 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:40:09.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:08 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:40:09.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:08 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:40:09.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:08 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:40:09.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:08 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-17T08:40:09.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:08 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-17T08:40:09.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:08 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-17T08:40:09.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:08 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:40:09.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:08 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:40:09.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:08 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T08:40:09.397 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:08 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:40:10.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:09 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:40:10.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:09 smithi118 ceph-mon[29491]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.y", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:40:10.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:09 smithi118 ceph-mon[29491]: Schedule rotate-key daemon mgr.y 2024-09-17T08:40:10.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:09 smithi118 ceph-mon[29491]: Rotating authentication key for mgr.y 2024-09-17T08:40:10.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:09 smithi118 ceph-mon[29491]: Reconfiguring daemon mgr.y on smithi118 2024-09-17T08:40:10.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:09 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:40:10.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:09 smithi118 ceph-mon[41418]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.y", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:40:10.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:09 smithi118 ceph-mon[41418]: Schedule rotate-key daemon mgr.y 2024-09-17T08:40:10.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:09 smithi118 ceph-mon[41418]: Rotating authentication key for mgr.y 2024-09-17T08:40:10.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:09 smithi118 ceph-mon[41418]: Reconfiguring daemon mgr.y on smithi118 2024-09-17T08:40:10.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:09 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:40:10.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:09 smithi170 ceph-mon[33271]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.y", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:40:10.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:09 smithi170 ceph-mon[33271]: Schedule rotate-key daemon mgr.y 2024-09-17T08:40:10.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:09 smithi170 ceph-mon[33271]: Rotating authentication key for mgr.y 2024-09-17T08:40:10.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:09 smithi170 ceph-mon[33271]: Reconfiguring daemon mgr.y on smithi118 2024-09-17T08:40:10.663 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.582+0000 7fce25253640 -1 mgr.server reply reply (13) Permission denied access denied: does your client key have mgr caps? See http://docs.ceph.com/en/latest/mgr/administrator/#client-authentication 2024-09-17T08:40:10.663 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.584+0000 7fce171b7640 -1 log_channel(cephadm) log [ERR] : Non-zero return from ['ceph', '-k', '/var/lib/ceph/mgr/ceph-y/keyring', '-n', 'mgr.y', 'tell', 'mgr.y', 'rotate-key', '-i', '-']: 2024-09-17T08:40:10.576+0000 7f23df50f640 1 Processor -- start 2024-09-17T08:40:10.663 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.576+0000 7f23df50f640 1 -- start start 2024-09-17T08:40:10.663 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.576+0000 7f23df50f640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f23d806ca60 0x7f23d806ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T08:40:10.663 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.576+0000 7f23df50f640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d806b360 con 0x7f23d806ca60 2024-09-17T08:40:10.663 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.577+0000 7f23dd2ab640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f23d806ca60 0x7f23d806ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T08:40:10.663 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.577+0000 7f23dd2ab640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f23d806ca60 0x7f23d806ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:34574/0 (socket says 172.21.15.118:34574) 2024-09-17T08:40:10.664 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.577+0000 7f23dd2ab640 1 -- 172.21.15.118:0/3267668045 learned_addr learned my addr 172.21.15.118:0/3267668045 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-09-17T08:40:10.664 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.577+0000 7f23dd2ab640 1 -- 172.21.15.118:0/3267668045 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d806b4a0 con 0x7f23d806ca60 2024-09-17T08:40:10.664 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.577+0000 7f23dd2ab640 1 --2- 172.21.15.118:0/3267668045 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f23d806ca60 0x7f23d806ae20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f23d4009820 tx=0x7f23d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ea374cbc00c9c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T08:40:10.664 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.578+0000 7f23cffff640 1 -- 172.21.15.118:0/3267668045 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d4032440 con 0x7f23d806ca60 2024-09-17T08:40:10.664 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.578+0000 7f23cffff640 1 -- 172.21.15.118:0/3267668045 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- auth(proto 2 2 bytes epoch 0) v1 -- 0x7f23c4003be0 con 0x7f23d806ca60 2024-09-17T08:40:10.664 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.578+0000 7f23cffff640 1 -- 172.21.15.118:0/3267668045 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(37 keys) v1 ==== 1740+0+0 (secure 0 0 0) 0x7f23d4032cd0 con 0x7f23d806ca60 2024-09-17T08:40:10.665 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.578+0000 7f23cffff640 1 -- 172.21.15.118:0/3267668045 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d4031880 con 0x7f23d806ca60 2024-09-17T08:40:10.665 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.578+0000 7f23cffff640 1 -- 172.21.15.118:0/3267668045 <== mon.0 v2:172.21.15.118:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x7f23c4003be0 con 0x7f23d806ca60 2024-09-17T08:40:10.665 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.578+0000 7f23cffff640 0 cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 8263963073697488557 != 18374858748799134293 2024-09-17T08:40:10.665 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.578+0000 7f23df50f640 1 -- 172.21.15.118:0/3267668045 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f23d806ca60 msgr2=0x7f23d806ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T08:40:10.665 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.578+0000 7f23df50f640 1 --2- 172.21.15.118:0/3267668045 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f23d806ca60 0x7f23d806ae20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f23d4009820 tx=0x7f23d402a270 comp rx=0 tx=0).stop 2024-09-17T08:40:10.665 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.578+0000 7f23df50f640 1 -- 172.21.15.118:0/3267668045 shutdown_connections 2024-09-17T08:40:10.665 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.578+0000 7f23df50f640 1 --2- 172.21.15.118:0/3267668045 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f23d806ca60 0x7f23d806ae20 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T08:40:10.665 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.578+0000 7f23df50f640 1 -- 172.21.15.118:0/3267668045 >> 172.21.15.118:0/3267668045 conn(0x7f23d806f430 msgr2=0x7f23d8071890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T08:40:10.666 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.578+0000 7f23df50f640 1 -- 172.21.15.118:0/3267668045 shutdown_connections 2024-09-17T08:40:10.666 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.578+0000 7f23df50f640 1 -- 172.21.15.118:0/3267668045 wait complete. 2024-09-17T08:40:10.666 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.579+0000 7f23df50f640 1 Processor -- start 2024-09-17T08:40:10.666 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.579+0000 7f23df50f640 1 -- start start 2024-09-17T08:40:10.666 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.579+0000 7f23df50f640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23d806ca60 0x7f23d81a2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T08:40:10.666 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.579+0000 7f23df50f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23d81a26b0 0x7f23d81a77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T08:40:10.666 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.579+0000 7f23df50f640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f23d81a2b30 0x7f23d81a9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T08:40:10.666 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.579+0000 7f23df50f640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d81aa4e0 con 0x7f23d81a2b30 2024-09-17T08:40:10.666 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.579+0000 7f23df50f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d81aa650 con 0x7f23d81a26b0 2024-09-17T08:40:10.666 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.579+0000 7f23df50f640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d81aa810 con 0x7f23d806ca60 2024-09-17T08:40:10.666 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.579+0000 7f23dd2ab640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23d806ca60 0x7f23d81a2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T08:40:10.666 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.579+0000 7f23ddaac640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f23d81a2b30 0x7f23d81a9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T08:40:10.667 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.579+0000 7f23ddaac640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f23d81a2b30 0x7f23d81a9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:34576/0 (socket says 172.21.15.118:34576) 2024-09-17T08:40:10.667 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.579+0000 7f23ddaac640 1 -- 172.21.15.118:0/1507154216 learned_addr learned my addr 172.21.15.118:0/1507154216 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-09-17T08:40:10.667 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.579+0000 7f23ddaac640 1 -- 172.21.15.118:0/1507154216 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23d806ca60 msgr2=0x7f23d81a2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T08:40:10.667 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.579+0000 7f23ddaac640 1 --2- 172.21.15.118:0/1507154216 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23d806ca60 0x7f23d81a2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T08:40:10.667 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.579+0000 7f23ddaac640 1 -- 172.21.15.118:0/1507154216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23d81a26b0 msgr2=0x7f23d81a77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T08:40:10.667 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.579+0000 7f23ddaac640 1 --2- 172.21.15.118:0/1507154216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23d81a26b0 0x7f23d81a77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T08:40:10.667 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.579+0000 7f23ddaac640 1 -- 172.21.15.118:0/1507154216 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d40094d0 con 0x7f23d81a2b30 2024-09-17T08:40:10.667 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.579+0000 7f23dd2ab640 1 --2- 172.21.15.118:0/1507154216 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23d806ca60 0x7f23d81a2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T08:40:10.667 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.580+0000 7f23ddaac640 1 --2- 172.21.15.118:0/1507154216 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f23d81a2b30 0x7f23d81a9d50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f23d000b9e0 tx=0x7f23d000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T08:40:10.667 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.580+0000 7f23ce7fc640 1 -- 172.21.15.118:0/1507154216 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d000cba0 con 0x7f23d81a2b30 2024-09-17T08:40:10.667 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.580+0000 7f23ce7fc640 1 -- 172.21.15.118:0/1507154216 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- auth(proto 2 2 bytes epoch 0) v1 -- 0x7f23c4003160 con 0x7f23d81a2b30 2024-09-17T08:40:10.667 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.580+0000 7f23ce7fc640 1 -- 172.21.15.118:0/1507154216 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(37 keys) v1 ==== 1740+0+0 (secure 0 0 0) 0x7f23d000cd00 con 0x7f23d81a2b30 2024-09-17T08:40:10.668 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.580+0000 7f23df50f640 1 -- 172.21.15.118:0/1507154216 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d81aaaf0 con 0x7f23d81a2b30 2024-09-17T08:40:10.668 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.580+0000 7f23ce7fc640 1 -- 172.21.15.118:0/1507154216 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d00125a0 con 0x7f23d81a2b30 2024-09-17T08:40:10.668 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.580+0000 7f23df50f640 1 -- 172.21.15.118:0/1507154216 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d81ab090 con 0x7f23d81a2b30 2024-09-17T08:40:10.668 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.580+0000 7f23ce7fc640 1 -- 172.21.15.118:0/1507154216 <== mon.0 v2:172.21.15.118:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x7f23d81ab090 con 0x7f23d81a2b30 2024-09-17T08:40:10.668 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.580+0000 7f23ce7fc640 0 cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 8263963073697488557 != 18374858748799134293 2024-09-17T08:40:10.668 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.581+0000 7f23ce7fc640 1 -- 172.21.15.118:0/1507154216 <== mon.0 v2:172.21.15.118:3300/0 5 ==== mgrmap(e 22) v1 ==== 90822+0+0 (secure 0 0 0) 0x7f23d0007350 con 0x7f23d81a2b30 2024-09-17T08:40:10.668 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.582+0000 7f23ce7fc640 1 --2- 172.21.15.118:0/1507154216 >> [v2:172.21.15.118:6800/989036832,v1:172.21.15.118:6801/989036832] conn(0x7f23c406d6c0 0x7f23c406fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T08:40:10.668 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.582+0000 7f23ce7fc640 1 -- 172.21.15.118:0/1507154216 --> [v2:172.21.15.118:6800/989036832,v1:172.21.15.118:6801/989036832] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f23c4070290 con 0x7f23c406d6c0 2024-09-17T08:40:10.668 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.582+0000 7f23ce7fc640 1 -- 172.21.15.118:0/1507154216 <== mon.0 v2:172.21.15.118:3300/0 6 ==== osd_map(71..71 src has 1..71) v4 ==== 7508+0+0 (secure 0 0 0) 0x7f23d008d420 con 0x7f23d81a2b30 2024-09-17T08:40:10.668 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.582+0000 7f23dd2ab640 1 --2- 172.21.15.118:0/1507154216 >> [v2:172.21.15.118:6800/989036832,v1:172.21.15.118:6801/989036832] conn(0x7f23c406d6c0 0x7f23c406fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T08:40:10.668 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.582+0000 7f23dd2ab640 1 --2- 172.21.15.118:0/1507154216 >> [v2:172.21.15.118:6800/989036832,v1:172.21.15.118:6801/989036832] conn(0x7f23c406d6c0 0x7f23c406fb80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f23d4000c00 tx=0x7f23d402c040 comp rx=0 tx=0).ready entity=mgr.14550 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T08:40:10.668 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.582+0000 7f23ce7fc640 1 -- 172.21.15.118:0/1507154216 <== mgr.14550 v2:172.21.15.118:6800/989036832 1 ==== command_reply(tid 0: -13 access denied: does your client key have mgr caps? See http://docs.ceph.com/en/latest/mgr/administrator/#client-authentication) v1 ==== 134+0+0 (secure 0 0 0) 0x7f23c4070290 con 0x7f23c406d6c0 2024-09-17T08:40:10.668 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: Error EACCES: problem getting command descriptions from mgr.y 2024-09-17T08:40:10.669 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.669 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.618+0000 7fce80b24640 -1 mgr handle_mgr_map I was active but no longer am 2024-09-17T08:40:10.669 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ignoring --setuser ceph since I am not root 2024-09-17T08:40:10.669 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ignoring --setgroup ceph since I am not root 2024-09-17T08:40:10.895 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:10 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: [17/Sep/2024:08:40:10] ENGINE Bus STOPPING 2024-09-17T08:40:10.917 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.755+0000 7fe34c92f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T08:40:10.917 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.810+0000 7fe34c92f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T08:40:11.172 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:10.959+0000 7fe34c92f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T08:40:11.172 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:11 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:11.025+0000 7fe34c92f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T08:40:11.173 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:11 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:11.171+0000 7fe34c92f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:40:11.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[41418]: pgmap v226: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:40:11.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:40:11.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:40:11.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[41418]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-17T08:40:11.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-17T08:40:11.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[41418]: Activating manager daemon x 2024-09-17T08:40:11.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[41418]: osdmap e72: 8 total, 8 up, 8 in 2024-09-17T08:40:11.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[41418]: from='mgr.14550 ' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-09-17T08:40:11.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[41418]: mgrmap e23: x(active, starting, since 0.00648281s) 2024-09-17T08:40:11.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:40:11.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:40:11.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:40:11.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-17T08:40:11.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:40:11.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:40:11.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:40:11.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:40:11.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:40:11.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:40:11.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:40:11.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:40:11.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-17T08:40:11.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-17T08:40:11.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-17T08:40:11.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[29491]: pgmap v226: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:40:11.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:40:11.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:40:11.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[29491]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-17T08:40:11.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-17T08:40:11.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[29491]: Activating manager daemon x 2024-09-17T08:40:11.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[29491]: osdmap e72: 8 total, 8 up, 8 in 2024-09-17T08:40:11.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[29491]: from='mgr.14550 ' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-09-17T08:40:11.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[29491]: mgrmap e23: x(active, starting, since 0.00648281s) 2024-09-17T08:40:11.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:40:11.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:40:11.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:40:11.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-17T08:40:11.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:40:11.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:40:11.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:40:11.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:40:11.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:40:11.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:40:11.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:40:11.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:40:11.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-17T08:40:11.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-17T08:40:11.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:10 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-17T08:40:11.186 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:11 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: [17/Sep/2024:08:40:11] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-17T08:40:11.186 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:11 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: [17/Sep/2024:08:40:11] ENGINE Bus STOPPED 2024-09-17T08:40:11.186 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:11 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: [17/Sep/2024:08:40:11] ENGINE Bus STARTING 2024-09-17T08:40:11.186 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:11 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: [17/Sep/2024:08:40:11] ENGINE Serving on http://:::9283 2024-09-17T08:40:11.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:10 smithi170 ceph-mon[33271]: pgmap v226: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:40:11.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:10 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:40:11.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:10 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' 2024-09-17T08:40:11.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:10 smithi170 ceph-mon[33271]: from='mgr.14550 172.21.15.118:0/1864968286' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-17T08:40:11.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:10 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-17T08:40:11.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:10 smithi170 ceph-mon[33271]: Activating manager daemon x 2024-09-17T08:40:11.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:10 smithi170 ceph-mon[33271]: osdmap e72: 8 total, 8 up, 8 in 2024-09-17T08:40:11.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:10 smithi170 ceph-mon[33271]: from='mgr.14550 ' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-09-17T08:40:11.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:10 smithi170 ceph-mon[33271]: mgrmap e23: x(active, starting, since 0.00648281s) 2024-09-17T08:40:11.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:10 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:40:11.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:10 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:40:11.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:10 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:40:11.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:10 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-17T08:40:11.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:10 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:40:11.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:10 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:40:11.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:10 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:40:11.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:10 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:40:11.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:10 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:40:11.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:10 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:40:11.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:10 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:40:11.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:10 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:40:11.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:10 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-17T08:40:11.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:10 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-17T08:40:11.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:10 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-17T08:40:11.527 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:11 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:11.312+0000 7fe34c92f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T08:40:11.645 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:11 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: [17/Sep/2024:08:40:11] ENGINE Bus STARTED 2024-09-17T08:40:11.899 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:11 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:11.640+0000 7fe34c92f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T08:40:11.899 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:11 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:11.709+0000 7fe34c92f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T08:40:11.899 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:11 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:11.898+0000 7fe34c92f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T08:40:12.192 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:11 smithi170 ceph-mon[33271]: Manager daemon x is now available 2024-09-17T08:40:12.192 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:11 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:40:12.192 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:11 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:40:12.193 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:11 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-17T08:40:12.193 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:11 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-17T08:40:12.193 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:11 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-17T08:40:12.193 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:11 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-17T08:40:12.241 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:11 smithi118 ceph-mon[29491]: Manager daemon x is now available 2024-09-17T08:40:12.241 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:11 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:40:12.241 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:11 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:40:12.241 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:11 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-17T08:40:12.241 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:11 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-17T08:40:12.241 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:11 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-17T08:40:12.241 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:11 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-17T08:40:12.242 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:11 smithi118 ceph-mon[41418]: Manager daemon x is now available 2024-09-17T08:40:12.242 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:11 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:40:12.242 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:11 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:40:12.242 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:11 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-17T08:40:12.242 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:11 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-17T08:40:12.242 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:11 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-17T08:40:12.242 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:11 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-17T08:40:13.119 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:12 smithi118 ceph-mon[29491]: mgrmap e24: x(active, since 1.31471s) 2024-09-17T08:40:13.120 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:12 smithi118 ceph-mon[29491]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:40:13.120 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:12 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:12.858+0000 7fe34c92f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T08:40:13.120 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:12 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:12.931+0000 7fe34c92f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T08:40:13.120 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:12 smithi118 ceph-mon[41418]: mgrmap e24: x(active, since 1.31471s) 2024-09-17T08:40:13.120 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:12 smithi118 ceph-mon[41418]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:40:13.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:12 smithi170 ceph-mon[33271]: mgrmap e24: x(active, since 1.31471s) 2024-09-17T08:40:13.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:12 smithi170 ceph-mon[33271]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:40:13.751 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key mgr.y 2024-09-17T08:40:13.851 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:13 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:13.583+0000 7fa68ff7e640 -1 log_channel(cephadm) log [ERR] : [17/Sep/2024:08:40:13] ENGINE Error in HTTPServer.serve 2024-09-17T08:40:13.851 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:13 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: Traceback (most recent call last): 2024-09-17T08:40:13.852 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:13 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T08:40:13.852 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:13 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: self._connections.run(self.expiration_interval) 2024-09-17T08:40:13.852 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:13 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T08:40:13.852 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:13 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: self._run(expiration_interval) 2024-09-17T08:40:13.852 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:13 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T08:40:13.852 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:13 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T08:40:13.852 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:13 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T08:40:13.852 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:13 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T08:40:13.852 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:13 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T08:40:13.852 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:13 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: s = self.context.wrap_socket( 2024-09-17T08:40:13.852 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:13 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T08:40:13.853 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:13 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: return self.sslsocket_class._create( 2024-09-17T08:40:13.853 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:13 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T08:40:13.853 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:13 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: self.do_handshake() 2024-09-17T08:40:13.853 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:13 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T08:40:13.853 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:13 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: self._sslobj.do_handshake() 2024-09-17T08:40:13.853 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:13 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T08:40:13.853 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:13 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:13.928 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:13 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:13.671+0000 7fe34c92f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T08:40:13.928 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:13 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:13.789+0000 7fe34c92f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T08:40:14.094 INFO:teuthology.orchestra.run.smithi118.stdout:rotating key for mgr.x 2024-09-17T08:40:14.094 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQBoQOlm8KHSMBAAqyYNLChz91kONSz8qIyfdg== 2024-09-17T08:40:14.095 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQA+Pelmu2hOOxAAXh7oN9t3ukF9M4Mmr9NejQ== == AQBoQOlm8KHSMBAAqyYNLChz91kONSz8qIyfdg== ']' 2024-09-17T08:40:14.095 INFO:teuthology.orchestra.run.smithi118.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-17T08:40:14.095 INFO:teuthology.orchestra.run.smithi118.stderr:+ echo 'rotating key for mgr.x' 2024-09-17T08:40:14.095 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key mgr.x 2024-09-17T08:40:14.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:13 smithi170 ceph-mon[33271]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:40:14.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:13 smithi170 ceph-mon[33271]: mgrmap e25: x(active, since 2s) 2024-09-17T08:40:14.235 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:13 smithi118 ceph-mon[29491]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:40:14.235 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:13 smithi118 ceph-mon[29491]: mgrmap e25: x(active, since 2s) 2024-09-17T08:40:14.236 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:13 smithi118 ceph-mon[41418]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:40:14.236 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:13 smithi118 ceph-mon[41418]: mgrmap e25: x(active, since 2s) 2024-09-17T08:40:14.236 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:13 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:13.944+0000 7fe34c92f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:40:14.236 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:14 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:14.008+0000 7fe34c92f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T08:40:14.236 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:14 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:14.162+0000 7fe34c92f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T08:40:14.236 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:14 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:14.234+0000 7fe34c92f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T08:40:14.421 INFO:teuthology.orchestra.run.smithi118.stderr:+ K=AQCyPelmMXsQMRAAuNrFaVE8W6lBz0fqO2ar6w== 2024-09-17T08:40:14.422 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQCyPelmMXsQMRAAuNrFaVE8W6lBz0fqO2ar6w== 2024-09-17T08:40:14.422 INFO:teuthology.orchestra.run.smithi118.stderr:+ ceph orch daemon rotate-key mgr.x 2024-09-17T08:40:14.527 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:14 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:14.296+0000 7fe34c92f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T08:40:14.527 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:14 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:14.460+0000 7fe34c92f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T08:40:14.714 INFO:teuthology.orchestra.run.smithi118.stdout:Scheduled to rotate-key mgr.x on host 'smithi170' 2024-09-17T08:40:14.724 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQCyPelmMXsQMRAAuNrFaVE8W6lBz0fqO2ar6w== == AQCyPelmMXsQMRAAuNrFaVE8W6lBz0fqO2ar6w== ']' 2024-09-17T08:40:14.725 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:40:14.821 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:14 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:14.532+0000 7fe34c92f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T08:40:14.821 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:14 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:14.607+0000 7fe34c92f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T08:40:14.821 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:14 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:14.693+0000 7fe34c92f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T08:40:14.821 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:14 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:14.758+0000 7fe34c92f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T08:40:14.821 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:14 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:14.820+0000 7fe34c92f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T08:40:15.269 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:14 smithi170 ceph-mon[33271]: [17/Sep/2024:08:40:13] ENGINE Bus STARTING 2024-09-17T08:40:15.269 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:14 smithi170 ceph-mon[33271]: [17/Sep/2024:08:40:13] ENGINE Serving on https://172.21.15.170:7150 2024-09-17T08:40:15.269 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:14 smithi170 ceph-mon[33271]: [17/Sep/2024:08:40:13] ENGINE Error in HTTPServer.serve 2024-09-17T08:40:15.269 INFO:journalctl@ceph.mon.b.smithi170.stdout: Traceback (most recent call last): 2024-09-17T08:40:15.270 INFO:journalctl@ceph.mon.b.smithi170.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T08:40:15.270 INFO:journalctl@ceph.mon.b.smithi170.stdout: self._connections.run(self.expiration_interval) 2024-09-17T08:40:15.270 INFO:journalctl@ceph.mon.b.smithi170.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T08:40:15.270 INFO:journalctl@ceph.mon.b.smithi170.stdout: self._run(expiration_interval) 2024-09-17T08:40:15.270 INFO:journalctl@ceph.mon.b.smithi170.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T08:40:15.270 INFO:journalctl@ceph.mon.b.smithi170.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T08:40:15.270 INFO:journalctl@ceph.mon.b.smithi170.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T08:40:15.270 INFO:journalctl@ceph.mon.b.smithi170.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T08:40:15.270 INFO:journalctl@ceph.mon.b.smithi170.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T08:40:15.270 INFO:journalctl@ceph.mon.b.smithi170.stdout: s = self.context.wrap_socket( 2024-09-17T08:40:15.270 INFO:journalctl@ceph.mon.b.smithi170.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T08:40:15.270 INFO:journalctl@ceph.mon.b.smithi170.stdout: return self.sslsocket_class._create( 2024-09-17T08:40:15.270 INFO:journalctl@ceph.mon.b.smithi170.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T08:40:15.270 INFO:journalctl@ceph.mon.b.smithi170.stdout: self.do_handshake() 2024-09-17T08:40:15.270 INFO:journalctl@ceph.mon.b.smithi170.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T08:40:15.271 INFO:journalctl@ceph.mon.b.smithi170.stdout: self._sslobj.do_handshake() 2024-09-17T08:40:15.271 INFO:journalctl@ceph.mon.b.smithi170.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T08:40:15.271 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:14 smithi170 ceph-mon[33271]: [17/Sep/2024:08:40:13] ENGINE Serving on http://172.21.15.170:8765 2024-09-17T08:40:15.271 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:14 smithi170 ceph-mon[33271]: [17/Sep/2024:08:40:13] ENGINE Bus STARTED 2024-09-17T08:40:15.271 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:14 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/542193029' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-17T08:40:15.271 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:14 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:15.271 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:14 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:15.271 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:14 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/2794136488' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-17T08:40:15.271 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:14 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:15.271 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:14 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:15.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:14 smithi118 ceph-mon[29491]: [17/Sep/2024:08:40:13] ENGINE Bus STARTING 2024-09-17T08:40:15.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:14 smithi118 ceph-mon[29491]: [17/Sep/2024:08:40:13] ENGINE Serving on https://172.21.15.170:7150 2024-09-17T08:40:15.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:14 smithi118 ceph-mon[29491]: [17/Sep/2024:08:40:13] ENGINE Error in HTTPServer.serve 2024-09-17T08:40:15.277 INFO:journalctl@ceph.mon.a.smithi118.stdout: Traceback (most recent call last): 2024-09-17T08:40:15.277 INFO:journalctl@ceph.mon.a.smithi118.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T08:40:15.277 INFO:journalctl@ceph.mon.a.smithi118.stdout: self._connections.run(self.expiration_interval) 2024-09-17T08:40:15.277 INFO:journalctl@ceph.mon.a.smithi118.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T08:40:15.278 INFO:journalctl@ceph.mon.a.smithi118.stdout: self._run(expiration_interval) 2024-09-17T08:40:15.278 INFO:journalctl@ceph.mon.a.smithi118.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T08:40:15.278 INFO:journalctl@ceph.mon.a.smithi118.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T08:40:15.278 INFO:journalctl@ceph.mon.a.smithi118.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T08:40:15.278 INFO:journalctl@ceph.mon.a.smithi118.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T08:40:15.278 INFO:journalctl@ceph.mon.a.smithi118.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T08:40:15.278 INFO:journalctl@ceph.mon.a.smithi118.stdout: s = self.context.wrap_socket( 2024-09-17T08:40:15.278 INFO:journalctl@ceph.mon.a.smithi118.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T08:40:15.278 INFO:journalctl@ceph.mon.a.smithi118.stdout: return self.sslsocket_class._create( 2024-09-17T08:40:15.278 INFO:journalctl@ceph.mon.a.smithi118.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T08:40:15.278 INFO:journalctl@ceph.mon.a.smithi118.stdout: self.do_handshake() 2024-09-17T08:40:15.278 INFO:journalctl@ceph.mon.a.smithi118.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T08:40:15.278 INFO:journalctl@ceph.mon.a.smithi118.stdout: self._sslobj.do_handshake() 2024-09-17T08:40:15.278 INFO:journalctl@ceph.mon.a.smithi118.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T08:40:15.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:14 smithi118 ceph-mon[29491]: [17/Sep/2024:08:40:13] ENGINE Serving on http://172.21.15.170:8765 2024-09-17T08:40:15.279 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:14 smithi118 ceph-mon[29491]: [17/Sep/2024:08:40:13] ENGINE Bus STARTED 2024-09-17T08:40:15.279 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:14 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/542193029' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-17T08:40:15.279 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:14 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:15.279 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:14 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:15.279 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:14 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/2794136488' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-17T08:40:15.279 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:14 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:15.279 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:14 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:15.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:14 smithi118 ceph-mon[41418]: [17/Sep/2024:08:40:13] ENGINE Bus STARTING 2024-09-17T08:40:15.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:14 smithi118 ceph-mon[41418]: [17/Sep/2024:08:40:13] ENGINE Serving on https://172.21.15.170:7150 2024-09-17T08:40:15.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:14 smithi118 ceph-mon[41418]: [17/Sep/2024:08:40:13] ENGINE Error in HTTPServer.serve 2024-09-17T08:40:15.279 INFO:journalctl@ceph.mon.c.smithi118.stdout: Traceback (most recent call last): 2024-09-17T08:40:15.279 INFO:journalctl@ceph.mon.c.smithi118.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T08:40:15.280 INFO:journalctl@ceph.mon.c.smithi118.stdout: self._connections.run(self.expiration_interval) 2024-09-17T08:40:15.280 INFO:journalctl@ceph.mon.c.smithi118.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T08:40:15.280 INFO:journalctl@ceph.mon.c.smithi118.stdout: self._run(expiration_interval) 2024-09-17T08:40:15.280 INFO:journalctl@ceph.mon.c.smithi118.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T08:40:15.280 INFO:journalctl@ceph.mon.c.smithi118.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T08:40:15.280 INFO:journalctl@ceph.mon.c.smithi118.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T08:40:15.280 INFO:journalctl@ceph.mon.c.smithi118.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T08:40:15.280 INFO:journalctl@ceph.mon.c.smithi118.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T08:40:15.280 INFO:journalctl@ceph.mon.c.smithi118.stdout: s = self.context.wrap_socket( 2024-09-17T08:40:15.280 INFO:journalctl@ceph.mon.c.smithi118.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T08:40:15.280 INFO:journalctl@ceph.mon.c.smithi118.stdout: return self.sslsocket_class._create( 2024-09-17T08:40:15.280 INFO:journalctl@ceph.mon.c.smithi118.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T08:40:15.280 INFO:journalctl@ceph.mon.c.smithi118.stdout: self.do_handshake() 2024-09-17T08:40:15.280 INFO:journalctl@ceph.mon.c.smithi118.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T08:40:15.280 INFO:journalctl@ceph.mon.c.smithi118.stdout: self._sslobj.do_handshake() 2024-09-17T08:40:15.281 INFO:journalctl@ceph.mon.c.smithi118.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T08:40:15.281 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:14 smithi118 ceph-mon[41418]: [17/Sep/2024:08:40:13] ENGINE Serving on http://172.21.15.170:8765 2024-09-17T08:40:15.281 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:14 smithi118 ceph-mon[41418]: [17/Sep/2024:08:40:13] ENGINE Bus STARTED 2024-09-17T08:40:15.281 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:14 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/542193029' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-17T08:40:15.281 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:14 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:15.281 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:14 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:15.281 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:14 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/2794136488' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-17T08:40:15.281 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:14 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:15.281 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:14 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:15.932 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:15.608+0000 7fe34c92f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T08:40:15.932 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: [17/Sep/2024:08:40:15] ENGINE Bus STARTING 2024-09-17T08:40:15.932 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: CherryPy Checker: 2024-09-17T08:40:15.932 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: The Application mounted at '' has an empty config. 2024-09-17T08:40:15.932 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: 2024-09-17T08:40:15.932 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: [17/Sep/2024:08:40:15] ENGINE Serving on http://:::9283 2024-09-17T08:40:15.933 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: [17/Sep/2024:08:40:15] ENGINE Bus STARTED 2024-09-17T08:40:16.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[29491]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:40:16.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[29491]: from='client.25235 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:40:16.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[29491]: Schedule rotate-key daemon mgr.x 2024-09-17T08:40:16.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:16.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[29491]: mgrmap e26: x(active, since 4s) 2024-09-17T08:40:16.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:16.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:16.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:16.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:16.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:16.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:16.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:16.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:16.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:16.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[29491]: Adjusting osd_memory_target on smithi118 to 3248M 2024-09-17T08:40:16.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:16.279 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:16.279 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:16.279 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[29491]: Standby manager daemon y started 2024-09-17T08:40:16.279 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[29491]: from='mgr.? 172.21.15.118:0/1713107704' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-17T08:40:16.279 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[29491]: from='mgr.? 172.21.15.118:0/1713107704' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T08:40:16.279 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[29491]: from='mgr.? 172.21.15.118:0/1713107704' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-17T08:40:16.279 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[29491]: from='mgr.? 172.21.15.118:0/1713107704' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T08:40:16.279 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[41418]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:40:16.280 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[41418]: from='client.25235 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:40:16.280 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[41418]: Schedule rotate-key daemon mgr.x 2024-09-17T08:40:16.280 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:16.280 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[41418]: mgrmap e26: x(active, since 4s) 2024-09-17T08:40:16.280 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:16.280 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:16.280 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:16.280 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:16.280 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:16.280 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:16.280 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:16.280 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:16.280 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:16.281 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[41418]: Adjusting osd_memory_target on smithi118 to 3248M 2024-09-17T08:40:16.281 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:16.281 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:16.281 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:16.281 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[41418]: Standby manager daemon y started 2024-09-17T08:40:16.281 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[41418]: from='mgr.? 172.21.15.118:0/1713107704' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-17T08:40:16.281 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[41418]: from='mgr.? 172.21.15.118:0/1713107704' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T08:40:16.281 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[41418]: from='mgr.? 172.21.15.118:0/1713107704' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-17T08:40:16.281 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:15 smithi118 ceph-mon[41418]: from='mgr.? 172.21.15.118:0/1713107704' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T08:40:16.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:15 smithi170 ceph-mon[33271]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:40:16.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:15 smithi170 ceph-mon[33271]: from='client.25235 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:40:16.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:15 smithi170 ceph-mon[33271]: Schedule rotate-key daemon mgr.x 2024-09-17T08:40:16.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:15 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:16.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:15 smithi170 ceph-mon[33271]: mgrmap e26: x(active, since 4s) 2024-09-17T08:40:16.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:15 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:16.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:15 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:16.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:15 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:16.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:15 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:16.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:15 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:16.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:15 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:16.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:15 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:16.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:15 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:16.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:15 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:16.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:15 smithi170 ceph-mon[33271]: Adjusting osd_memory_target on smithi118 to 3248M 2024-09-17T08:40:16.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:15 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:16.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:15 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:16.397 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:15 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:16.397 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:15 smithi170 ceph-mon[33271]: Standby manager daemon y started 2024-09-17T08:40:16.397 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:15 smithi170 ceph-mon[33271]: from='mgr.? 172.21.15.118:0/1713107704' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-17T08:40:16.397 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:15 smithi170 ceph-mon[33271]: from='mgr.? 172.21.15.118:0/1713107704' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T08:40:16.397 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:15 smithi170 ceph-mon[33271]: from='mgr.? 172.21.15.118:0/1713107704' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-17T08:40:16.397 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:15 smithi170 ceph-mon[33271]: from='mgr.? 172.21.15.118:0/1713107704' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T08:40:17.360 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[29743]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:40:17] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-17T08:40:17.360 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:17.360 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:17.360 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:17.360 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:17.360 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:17.361 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:17.361 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:17.361 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:17.361 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:17.361 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:17.361 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[29491]: Adjusting osd_memory_target on smithi170 to 3248M 2024-09-17T08:40:17.361 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:17.361 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:40:17.361 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:40:17.361 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[29491]: Updating smithi118:/etc/ceph/ceph.conf 2024-09-17T08:40:17.361 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[29491]: Updating smithi170:/etc/ceph/ceph.conf 2024-09-17T08:40:17.361 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[29491]: mgrmap e27: x(active, since 5s), standbys: y 2024-09-17T08:40:17.361 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-17T08:40:17.361 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:17.362 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:17.362 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:17.362 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:17.362 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:17.362 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-17T08:40:17.362 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-17T08:40:17.362 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-17T08:40:17.362 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:40:17.362 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:40:17.362 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T08:40:17.362 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:40:17.363 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:17.363 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:17.363 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:17.363 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:17.363 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:17.363 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:17.363 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:17.363 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:17.363 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:17.363 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:17.363 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[41418]: Adjusting osd_memory_target on smithi170 to 3248M 2024-09-17T08:40:17.363 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:17.363 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:40:17.363 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:40:17.364 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[41418]: Updating smithi118:/etc/ceph/ceph.conf 2024-09-17T08:40:17.364 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[41418]: Updating smithi170:/etc/ceph/ceph.conf 2024-09-17T08:40:17.364 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[41418]: mgrmap e27: x(active, since 5s), standbys: y 2024-09-17T08:40:17.364 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-17T08:40:17.364 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:17.364 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:17.364 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:17.364 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:17.364 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:17.364 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-17T08:40:17.364 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-17T08:40:17.364 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-17T08:40:17.364 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:40:17.364 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:40:17.365 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T08:40:17.365 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:17 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:40:17.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:17 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:17.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:17 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:17.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:17 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:17.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:17 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:17.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:17 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:17.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:17 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:17.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:17 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:17.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:17 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:17.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:17 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:17.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:17 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:40:17.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:17 smithi170 ceph-mon[33271]: Adjusting osd_memory_target on smithi170 to 3248M 2024-09-17T08:40:17.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:17 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:17.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:17 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:40:17.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:17 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:40:17.397 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:17 smithi170 ceph-mon[33271]: Updating smithi118:/etc/ceph/ceph.conf 2024-09-17T08:40:17.397 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:17 smithi170 ceph-mon[33271]: Updating smithi170:/etc/ceph/ceph.conf 2024-09-17T08:40:17.397 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:17 smithi170 ceph-mon[33271]: mgrmap e27: x(active, since 5s), standbys: y 2024-09-17T08:40:17.397 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:17 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-17T08:40:17.397 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:17 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:17.397 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:17 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:17.397 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:17 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:17.397 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:17 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:17.397 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:17 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:17.397 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:17 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-17T08:40:17.397 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:17 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-17T08:40:17.397 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:17 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-17T08:40:17.397 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:17 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:40:17.397 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:17 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:40:17.398 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:17 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T08:40:17.398 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:17 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:40:18.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:18 smithi170 ceph-mon[33271]: Updating smithi170:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:40:18.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:18 smithi170 ceph-mon[33271]: Updating smithi118:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:40:18.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:18 smithi170 ceph-mon[33271]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:40:18.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:18 smithi170 ceph-mon[33271]: Updating smithi118:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:40:18.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:18 smithi170 ceph-mon[33271]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:40:18.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:18 smithi170 ceph-mon[33271]: Updating smithi170:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:40:18.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:18 smithi170 ceph-mon[33271]: Updating smithi118:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:40:18.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:18 smithi170 ceph-mon[33271]: Rotating authentication key for mgr.y 2024-09-17T08:40:18.396 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:18 smithi170 ceph-mon[33271]: Reconfiguring daemon mgr.y on smithi118 2024-09-17T08:40:18.444 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:18 smithi118 ceph-mon[29491]: Updating smithi170:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:40:18.445 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:18 smithi118 ceph-mon[29491]: Updating smithi118:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:40:18.445 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:18 smithi118 ceph-mon[29491]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:40:18.445 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:18 smithi118 ceph-mon[29491]: Updating smithi118:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:40:18.445 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:18 smithi118 ceph-mon[29491]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:40:18.445 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:18 smithi118 ceph-mon[29491]: Updating smithi170:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:40:18.445 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:18 smithi118 ceph-mon[29491]: Updating smithi118:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:40:18.446 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:18 smithi118 ceph-mon[29491]: Rotating authentication key for mgr.y 2024-09-17T08:40:18.446 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:18 smithi118 ceph-mon[29491]: Reconfiguring daemon mgr.y on smithi118 2024-09-17T08:40:18.446 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:18 smithi118 ceph-mon[41418]: Updating smithi170:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:40:18.446 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:18 smithi118 ceph-mon[41418]: Updating smithi118:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:40:18.446 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:18 smithi118 ceph-mon[41418]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:40:18.447 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:18 smithi118 ceph-mon[41418]: Updating smithi118:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:40:18.447 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:18 smithi118 ceph-mon[41418]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:40:18.447 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:18 smithi118 ceph-mon[41418]: Updating smithi170:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:40:18.447 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:18 smithi118 ceph-mon[41418]: Updating smithi118:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:40:18.447 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:18 smithi118 ceph-mon[41418]: Rotating authentication key for mgr.y 2024-09-17T08:40:18.447 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:18 smithi118 ceph-mon[41418]: Reconfiguring daemon mgr.y on smithi118 2024-09-17T08:40:18.895 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:40:18 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:40:19.277 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:19 smithi118 systemd[1]: Stopping Ceph mgr.y for 7862d192-74ce-11ef-bceb-c7b262605968... 2024-09-17T08:40:19.574 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:19 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:19.574 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:19 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:19.574 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:19 smithi118 ceph-mon[29491]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 32 KiB/s rd, 0 B/s wr, 18 op/s 2024-09-17T08:40:19.574 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:19 smithi118 ceph-mon[29491]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:40:19.575 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:19 smithi118 podman[86267]: 2024-09-17 08:40:19.30572548 +0000 UTC m=+0.140040255 container died e3931710a4691a3fbe1ae831c8db36f36640f304601d01196f4efa46cbceca09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y, RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T08:40:19.575 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:19 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:19.575 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:19 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:19.575 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:19 smithi118 ceph-mon[41418]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 32 KiB/s rd, 0 B/s wr, 18 op/s 2024-09-17T08:40:19.575 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:19 smithi118 ceph-mon[41418]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:40:19.727 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key mgr.x 2024-09-17T08:40:19.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:19 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:19.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:19 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:19.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:19 smithi170 ceph-mon[33271]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 32 KiB/s rd, 0 B/s wr, 18 op/s 2024-09-17T08:40:19.896 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:19 smithi170 ceph-mon[33271]: from='client.24403 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:40:19.982 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:19 smithi118 podman[86267]: 2024-09-17 08:40:19.672715519 +0000 UTC m=+0.507030294 container cleanup e3931710a4691a3fbe1ae831c8db36f36640f304601d01196f4efa46cbceca09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y, io.buildah.version=1.37.2, RELEASE=reef-19e2a2b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-17T08:40:19.982 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:19 smithi118 bash[86267]: ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y 2024-09-17T08:40:20.062 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQCyPelmMXsQMRAAuNrFaVE8W6lBz0fqO2ar6w== 2024-09-17T08:40:20.062 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQCyPelmMXsQMRAAuNrFaVE8W6lBz0fqO2ar6w== == AQCyPelmMXsQMRAAuNrFaVE8W6lBz0fqO2ar6w== ']' 2024-09-17T08:40:20.063 INFO:teuthology.orchestra.run.smithi118.stderr:+ sleep 5 2024-09-17T08:40:20.277 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:19 smithi118 podman[86284]: 2024-09-17 08:40:19.982729175 +0000 UTC m=+0.674604725 container remove e3931710a4691a3fbe1ae831c8db36f36640f304601d01196f4efa46cbceca09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=reef-19e2a2b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True) 2024-09-17T08:40:20.277 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:19 smithi118 systemd[1]: ceph-7862d192-74ce-11ef-bceb-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-17T08:40:20.277 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:20 smithi118 systemd[1]: ceph-7862d192-74ce-11ef-bceb-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-09-17T08:40:20.277 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:20 smithi118 systemd[1]: Stopped Ceph mgr.y for 7862d192-74ce-11ef-bceb-c7b262605968. 2024-09-17T08:40:20.278 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:20 smithi118 systemd[1]: ceph-7862d192-74ce-11ef-bceb-c7b262605968@mgr.y.service: Consumed 47.555s CPU time. 2024-09-17T08:40:20.278 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:20 smithi118 systemd[1]: Starting Ceph mgr.y for 7862d192-74ce-11ef-bceb-c7b262605968... 2024-09-17T08:40:20.660 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:20 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/1393302814' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-17T08:40:20.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:20 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/1393302814' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-17T08:40:20.661 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:20 smithi118 podman[86405]: 2024-09-17 08:40:20.255537312 +0000 UTC m=+0.015628518 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-17T08:40:20.661 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:20 smithi118 podman[86405]: 2024-09-17 08:40:20.392183059 +0000 UTC m=+0.152274252 container create ce4a9a4bb346d095e63fb3ca52089c1a12e39f92cacf35fa0c2f40720d720636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-17T08:40:20.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:20 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/1393302814' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-17T08:40:20.992 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:20 smithi118 podman[86405]: 2024-09-17 08:40:20.661209242 +0000 UTC m=+0.421300436 container init ce4a9a4bb346d095e63fb3ca52089c1a12e39f92cacf35fa0c2f40720d720636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True) 2024-09-17T08:40:20.992 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:20 smithi118 podman[86405]: 2024-09-17 08:40:20.665752393 +0000 UTC m=+0.425843587 container start ce4a9a4bb346d095e63fb3ca52089c1a12e39f92cacf35fa0c2f40720d720636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-17T08:40:20.992 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:20 smithi118 ceph-mgr[86445]: -- 172.21.15.118:0/4161164467 <== mon.0 v2:172.21.15.118:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56188457cd00 con 0x561884550800 2024-09-17T08:40:20.992 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:20 smithi118 ceph-mgr[86445]: cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 13451154301200932091 != 18374858748799134293 2024-09-17T08:40:20.992 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:20 smithi118 bash[86405]: ce4a9a4bb346d095e63fb3ca52089c1a12e39f92cacf35fa0c2f40720d720636 2024-09-17T08:40:20.992 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:20 smithi118 systemd[1]: Started Ceph mgr.y for 7862d192-74ce-11ef-bceb-c7b262605968. 2024-09-17T08:40:20.992 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:20 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: 2024-09-17T08:40:20.783+0000 7f5d8294a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T08:40:20.992 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:20 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: 2024-09-17T08:40:20.842+0000 7f5d8294a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T08:40:20.992 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:20 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: 2024-09-17T08:40:20.991+0000 7f5d8294a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T08:40:21.277 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:21 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: 2024-09-17T08:40:21.057+0000 7f5d8294a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T08:40:21.277 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:21 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: 2024-09-17T08:40:21.203+0000 7f5d8294a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:40:21.640 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:21 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: 2024-09-17T08:40:21.342+0000 7f5d8294a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T08:40:21.897 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:40:21 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a[79253]: ts=2024-09-17T08:40:21.640Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi118.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.118:8443: connect: connection refused" 2024-09-17T08:40:21.898 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:21 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: 2024-09-17T08:40:21.654+0000 7f5d8294a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T08:40:21.898 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:21 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: 2024-09-17T08:40:21.714+0000 7f5d8294a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T08:40:21.898 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:21 smithi118 ceph-mon[29491]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2024-09-17T08:40:21.898 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:21 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:21.898 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:21 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:21.898 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:21 smithi118 ceph-mon[29491]: Rotating authentication key for mgr.x 2024-09-17T08:40:21.898 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:21 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-17T08:40:21.898 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:21 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-17T08:40:21.899 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:21 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]': finished 2024-09-17T08:40:21.899 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:21 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:40:21.899 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:21 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:40:21.899 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:21 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T08:40:21.899 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:21 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:40:21.899 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:21 smithi118 ceph-mon[29491]: Reconfiguring daemon mgr.x on smithi170 2024-09-17T08:40:21.899 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:21 smithi118 ceph-mon[41418]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2024-09-17T08:40:21.899 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:21 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:21.899 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:21 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:21.899 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:21 smithi118 ceph-mon[41418]: Rotating authentication key for mgr.x 2024-09-17T08:40:21.900 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:21 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-17T08:40:21.900 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:21 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-17T08:40:21.900 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:21 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]': finished 2024-09-17T08:40:21.900 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:21 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:40:21.900 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:21 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:40:21.900 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:21 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T08:40:21.900 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:21 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:40:21.900 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:21 smithi118 ceph-mon[41418]: Reconfiguring daemon mgr.x on smithi170 2024-09-17T08:40:22.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:21 smithi170 ceph-mon[33271]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2024-09-17T08:40:22.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:21 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:22.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:21 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:22.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:21 smithi170 ceph-mon[33271]: Rotating authentication key for mgr.x 2024-09-17T08:40:22.145 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:21 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-17T08:40:22.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:21 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-17T08:40:22.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:21 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]': finished 2024-09-17T08:40:22.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:21 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:40:22.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:21 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:40:22.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:21 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T08:40:22.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:21 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:40:22.146 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:21 smithi170 ceph-mon[33271]: Reconfiguring daemon mgr.x on smithi170 2024-09-17T08:40:22.277 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:21 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: 2024-09-17T08:40:21.896+0000 7f5d8294a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T08:40:22.603 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.450+0000 7fa6c97f1640 -1 mgr.server reply reply (13) Permission denied access denied: does your client key have mgr caps? See http://docs.ceph.com/en/latest/mgr/administrator/#client-authentication 2024-09-17T08:40:22.604 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.452+0000 7fa6c6fec640 -1 log_channel(cephadm) log [ERR] : Non-zero return from ['ceph', '-k', '/var/lib/ceph/mgr/ceph-x/keyring', '-n', 'mgr.x', 'tell', 'mgr.x', 'rotate-key', '-i', '-']: 2024-09-17T08:40:22.445+0000 7feb0cfac640 1 -- 172.21.15.170:0/642572708 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x7feaf4003750 con 0x7feb08108790 2024-09-17T08:40:22.604 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.445+0000 7feb0cfac640 0 cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 4496123844612982110 != 18374858748799134293 2024-09-17T08:40:22.604 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.446+0000 7feb10212640 1 -- 172.21.15.170:0/642572708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb08108790 msgr2=0x7feb08108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T08:40:22.604 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.446+0000 7feb10212640 1 --2- 172.21.15.170:0/642572708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb08108790 0x7feb08108c10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7feaf8009900 tx=0x7feaf802a410 comp rx=0 tx=0).stop 2024-09-17T08:40:22.604 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.446+0000 7feb10212640 1 -- 172.21.15.170:0/642572708 shutdown_connections 2024-09-17T08:40:22.604 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.446+0000 7feb10212640 1 --2- 172.21.15.170:0/642572708 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feb08109150 0x7feb0810fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T08:40:22.604 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.446+0000 7feb10212640 1 --2- 172.21.15.170:0/642572708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb08108790 0x7feb08108c10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T08:40:22.605 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.446+0000 7feb10212640 1 --2- 172.21.15.170:0/642572708 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feb08107840 0x7feb08107c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T08:40:22.605 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.446+0000 7feb10212640 1 -- 172.21.15.170:0/642572708 >> 172.21.15.170:0/642572708 conn(0x7feb0806dfa0 msgr2=0x7feb08070400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T08:40:22.605 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.446+0000 7feb10212640 1 -- 172.21.15.170:0/642572708 shutdown_connections 2024-09-17T08:40:22.605 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.446+0000 7feb10212640 1 -- 172.21.15.170:0/642572708 wait complete. 2024-09-17T08:40:22.605 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.446+0000 7feb10212640 1 Processor -- start 2024-09-17T08:40:22.605 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.447+0000 7feb10212640 1 -- start start 2024-09-17T08:40:22.606 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.447+0000 7feb10212640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb08107840 0x7feb0819ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T08:40:22.606 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.447+0000 7feb10212640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feb08108790 0x7feb081a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T08:40:22.606 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.447+0000 7feb10212640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feb08109150 0x7feb081a7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T08:40:22.606 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.447+0000 7feb10212640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb081a7ab0 con 0x7feb08108790 2024-09-17T08:40:22.606 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.447+0000 7feb10212640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb081a7c20 con 0x7feb08107840 2024-09-17T08:40:22.606 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.447+0000 7feb10212640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb081a7de0 con 0x7feb08109150 2024-09-17T08:40:22.606 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.447+0000 7feb0dfae640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb08107840 0x7feb0819ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T08:40:22.607 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.447+0000 7feb0dfae640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb08107840 0x7feb0819ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41608/0 (socket says 172.21.15.170:41608) 2024-09-17T08:40:22.607 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.447+0000 7feb0dfae640 1 -- 172.21.15.170:0/1182972211 learned_addr learned my addr 172.21.15.170:0/1182972211 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-17T08:40:22.607 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.447+0000 7feb0e7af640 1 --2- 172.21.15.170:0/1182972211 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feb08109150 0x7feb081a7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T08:40:22.607 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.447+0000 7feb0d7ad640 1 --2- 172.21.15.170:0/1182972211 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feb08108790 0x7feb081a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T08:40:22.607 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.447+0000 7feb0dfae640 1 -- 172.21.15.170:0/1182972211 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feb08109150 msgr2=0x7feb081a7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T08:40:22.607 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.447+0000 7feb0dfae640 1 --2- 172.21.15.170:0/1182972211 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feb08109150 0x7feb081a7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T08:40:22.607 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.447+0000 7feb0dfae640 1 -- 172.21.15.170:0/1182972211 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feb08108790 msgr2=0x7feb081a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T08:40:22.608 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.447+0000 7feb0dfae640 1 --2- 172.21.15.170:0/1182972211 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feb08108790 0x7feb081a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T08:40:22.608 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.447+0000 7feb0dfae640 1 -- 172.21.15.170:0/1182972211 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf8009560 con 0x7feb08107840 2024-09-17T08:40:22.608 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.447+0000 7feb0dfae640 1 --2- 172.21.15.170:0/1182972211 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb08107840 0x7feb0819ff00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7feb0400d950 tx=0x7feb0400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T08:40:22.608 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.447+0000 7feafeffd640 1 -- 172.21.15.170:0/1182972211 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb04009c30 con 0x7feb08107840 2024-09-17T08:40:22.608 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.447+0000 7feafeffd640 1 -- 172.21.15.170:0/1182972211 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- auth(proto 2 2 bytes epoch 0) v1 -- 0x7feaf4003310 con 0x7feb08107840 2024-09-17T08:40:22.608 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.447+0000 7feafeffd640 1 -- 172.21.15.170:0/1182972211 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(37 keys) v1 ==== 1740+0+0 (secure 0 0 0) 0x7feb04009d90 con 0x7feb08107840 2024-09-17T08:40:22.608 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.447+0000 7feb10212640 1 -- 172.21.15.170:0/1182972211 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb081a80c0 con 0x7feb08107840 2024-09-17T08:40:22.609 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.447+0000 7feafeffd640 1 -- 172.21.15.170:0/1182972211 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0400e5b0 con 0x7feb08107840 2024-09-17T08:40:22.609 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.448+0000 7feb10212640 1 -- 172.21.15.170:0/1182972211 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb081a8590 con 0x7feb08107840 2024-09-17T08:40:22.609 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.448+0000 7feafeffd640 1 -- 172.21.15.170:0/1182972211 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x7feb081a8590 con 0x7feb08107840 2024-09-17T08:40:22.609 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.448+0000 7feafeffd640 0 cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 4496123844612982110 != 18374858748799134293 2024-09-17T08:40:22.609 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.449+0000 7feafeffd640 1 -- 172.21.15.170:0/1182972211 <== mon.1 v2:172.21.15.170:3300/0 5 ==== mgrmap(e 27) v1 ==== 90822+0+0 (secure 0 0 0) 0x7feb0400f3c0 con 0x7feb08107840 2024-09-17T08:40:22.609 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.449+0000 7feafeffd640 1 --2- 172.21.15.170:0/1182972211 >> [v2:172.21.15.170:6832/2557292516,v1:172.21.15.170:6833/2557292516] conn(0x7feaf406d310 0x7feaf406f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T08:40:22.609 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.449+0000 7feafeffd640 1 -- 172.21.15.170:0/1182972211 --> [v2:172.21.15.170:6832/2557292516,v1:172.21.15.170:6833/2557292516] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7feaf406fee0 con 0x7feaf406d310 2024-09-17T08:40:22.610 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.449+0000 7feafeffd640 1 -- 172.21.15.170:0/1182972211 <== mon.1 v2:172.21.15.170:3300/0 6 ==== osd_map(72..72 src has 1..72) v4 ==== 7766+0+0 (secure 0 0 0) 0x7feb04021030 con 0x7feb08107840 2024-09-17T08:40:22.610 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.449+0000 7feb0d7ad640 1 --2- 172.21.15.170:0/1182972211 >> [v2:172.21.15.170:6832/2557292516,v1:172.21.15.170:6833/2557292516] conn(0x7feaf406d310 0x7feaf406f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T08:40:22.610 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.450+0000 7feb0d7ad640 1 --2- 172.21.15.170:0/1182972211 >> [v2:172.21.15.170:6832/2557292516,v1:172.21.15.170:6833/2557292516] conn(0x7feaf406d310 0x7feaf406f7d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7feaf8009850 tx=0x7feaf8038000 comp rx=0 tx=0).ready entity=mgr.24448 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T08:40:22.610 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.450+0000 7feafeffd640 1 -- 172.21.15.170:0/1182972211 <== mgr.24448 v2:172.21.15.170:6832/2557292516 1 ==== command_reply(tid 0: -13 access denied: does your client key have mgr caps? See http://docs.ceph.com/en/latest/mgr/administrator/#client-authentication) v1 ==== 134+0+0 (secure 0 0 0) 0x7feaf406fee0 con 0x7feaf406d310 2024-09-17T08:40:22.610 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: Error EACCES: problem getting command descriptions from mgr.x 2024-09-17T08:40:22.610 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.610 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.462+0000 7fa73139a640 -1 mgr handle_mgr_map I was active but no longer am 2024-09-17T08:40:22.611 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: ignoring --setuser ceph since I am not root 2024-09-17T08:40:22.611 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: ignoring --setgroup ceph since I am not root 2024-09-17T08:40:22.611 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-mgr[35374]: -- 172.21.15.170:0/2582187001 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5599195f8d00 con 0x5599195cc800 2024-09-17T08:40:22.611 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-mgr[35374]: cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 4496123844612982110 != 18374858748799134293 2024-09-17T08:40:22.611 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.601+0000 7fce9147c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T08:40:22.872 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.659+0000 7fce9147c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T08:40:22.873 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.807+0000 7fce9147c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T08:40:23.145 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:22.872+0000 7fce9147c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T08:40:23.145 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:23 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:23.017+0000 7fce9147c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:40:23.277 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:22 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: 2024-09-17T08:40:22.776+0000 7f5d8294a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T08:40:23.277 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:22 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: 2024-09-17T08:40:22.841+0000 7f5d8294a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T08:40:23.461 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:23 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:23.155+0000 7fce9147c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T08:40:23.461 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:23 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:23.461 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:23 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:23.461 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:23 smithi170 ceph-mon[33271]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-17T08:40:23.462 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:23 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-17T08:40:23.462 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:23 smithi170 ceph-mon[33271]: Activating manager daemon y 2024-09-17T08:40:23.462 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:23 smithi170 ceph-mon[33271]: osdmap e73: 8 total, 8 up, 8 in 2024-09-17T08:40:23.462 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:23 smithi170 ceph-mon[33271]: from='mgr.24448 ' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2024-09-17T08:40:23.462 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:23 smithi170 ceph-mon[33271]: mgrmap e28: y(active, starting, since 0.00601939s) 2024-09-17T08:40:23.662 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:23 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: 2024-09-17T08:40:23.550+0000 7f5d8294a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T08:40:23.663 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:23 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:23.663 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:23 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:23.663 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:23 smithi118 ceph-mon[29491]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-17T08:40:23.663 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:23 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-17T08:40:23.663 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:23 smithi118 ceph-mon[29491]: Activating manager daemon y 2024-09-17T08:40:23.663 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:23 smithi118 ceph-mon[29491]: osdmap e73: 8 total, 8 up, 8 in 2024-09-17T08:40:23.663 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:23 smithi118 ceph-mon[29491]: from='mgr.24448 ' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2024-09-17T08:40:23.663 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:23 smithi118 ceph-mon[29491]: mgrmap e28: y(active, starting, since 0.00601939s) 2024-09-17T08:40:23.663 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:23 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:23.664 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:23 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' 2024-09-17T08:40:23.664 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:23 smithi118 ceph-mon[41418]: from='mgr.24448 172.21.15.170:0/2597954634' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-17T08:40:23.664 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:23 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-17T08:40:23.664 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:23 smithi118 ceph-mon[41418]: Activating manager daemon y 2024-09-17T08:40:23.664 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:23 smithi118 ceph-mon[41418]: osdmap e73: 8 total, 8 up, 8 in 2024-09-17T08:40:23.664 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:23 smithi118 ceph-mon[41418]: from='mgr.24448 ' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2024-09-17T08:40:23.664 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:23 smithi118 ceph-mon[41418]: mgrmap e28: y(active, starting, since 0.00601939s) 2024-09-17T08:40:23.895 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:23 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:23.460+0000 7fce9147c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T08:40:23.895 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:23 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:23.521+0000 7fce9147c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T08:40:23.895 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:23 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:23.703+0000 7fce9147c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T08:40:24.006 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:23 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: 2024-09-17T08:40:23.662+0000 7f5d8294a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T08:40:24.006 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:23 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: 2024-09-17T08:40:23.801+0000 7f5d8294a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:40:24.006 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:23 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: 2024-09-17T08:40:23.861+0000 7f5d8294a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T08:40:24.006 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:24 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: 2024-09-17T08:40:24.005+0000 7f5d8294a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T08:40:24.277 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:24 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: 2024-09-17T08:40:24.073+0000 7f5d8294a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T08:40:24.277 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:24 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: 2024-09-17T08:40:24.131+0000 7f5d8294a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T08:40:24.553 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:24 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: 2024-09-17T08:40:24.284+0000 7f5d8294a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T08:40:24.553 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:24 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: 2024-09-17T08:40:24.347+0000 7f5d8294a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T08:40:24.553 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:24 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: 2024-09-17T08:40:24.412+0000 7f5d8294a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T08:40:24.553 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:24 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: 2024-09-17T08:40:24.493+0000 7f5d8294a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T08:40:24.895 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:24 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:24.569+0000 7fce9147c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T08:40:24.895 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:24 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:24.635+0000 7fce9147c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T08:40:25.027 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:24 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: 2024-09-17T08:40:24.552+0000 7f5d8294a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T08:40:25.027 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:24 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: 2024-09-17T08:40:24.610+0000 7f5d8294a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T08:40:25.065 INFO:teuthology.orchestra.run.smithi118.stderr:++ ceph auth get-key mgr.x 2024-09-17T08:40:25.386 INFO:teuthology.orchestra.run.smithi118.stderr:+ NK=AQB0QOlmJq2RMBAAJ81Shm0fWHPxR1sJnMIBpQ== 2024-09-17T08:40:25.387 INFO:teuthology.orchestra.run.smithi118.stderr:+ '[' AQCyPelmMXsQMRAAuNrFaVE8W6lBz0fqO2ar6w== == AQB0QOlmJq2RMBAAJ81Shm0fWHPxR1sJnMIBpQ== ']' 2024-09-17T08:40:25.600 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:25 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:25.341+0000 7fce9147c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T08:40:25.600 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:25 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:25.459+0000 7fce9147c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T08:40:25.600 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:25 smithi170 ceph-mon[33271]: from='client.? 172.21.15.118:0/3319445639' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-17T08:40:25.600 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:25 smithi170 ceph-mon[33271]: Active manager daemon y restarted 2024-09-17T08:40:25.600 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:25 smithi170 ceph-mon[33271]: Activating manager daemon y 2024-09-17T08:40:25.600 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:25 smithi170 ceph-mon[33271]: osdmap e74: 8 total, 8 up, 8 in 2024-09-17T08:40:25.601 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:25 smithi170 ceph-mon[33271]: mgrmap e29: y(active, starting, since 0.0054385s) 2024-09-17T08:40:25.601 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:25 smithi170 ceph-mon[33271]: from='mgr.? 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-17T08:40:25.601 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:25 smithi170 ceph-mon[33271]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-17T08:40:25.601 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:25 smithi170 ceph-mon[33271]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:40:25.601 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:25 smithi170 ceph-mon[33271]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:40:25.601 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:25 smithi170 ceph-mon[33271]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:40:25.601 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:25 smithi170 ceph-mon[33271]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-17T08:40:25.601 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:25 smithi170 ceph-mon[33271]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:40:25.601 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:25 smithi170 ceph-mon[33271]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:40:25.601 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:25 smithi170 ceph-mon[33271]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:40:25.601 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:25 smithi170 ceph-mon[33271]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:40:25.601 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:25 smithi170 ceph-mon[33271]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:40:25.601 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:25 smithi170 ceph-mon[33271]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:40:25.602 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:25 smithi170 ceph-mon[33271]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:40:25.602 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:25 smithi170 ceph-mon[33271]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:40:25.602 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:25 smithi170 ceph-mon[33271]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-17T08:40:25.602 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:25 smithi170 ceph-mon[33271]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-17T08:40:25.602 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:25 smithi170 ceph-mon[33271]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-17T08:40:25.602 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:25 smithi170 ceph-mon[33271]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T08:40:25.602 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:25 smithi170 ceph-mon[33271]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-17T08:40:25.602 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:25 smithi170 ceph-mon[33271]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T08:40:25.640 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[29491]: from='client.? 172.21.15.118:0/3319445639' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-17T08:40:25.640 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[29491]: Active manager daemon y restarted 2024-09-17T08:40:25.640 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[29491]: Activating manager daemon y 2024-09-17T08:40:25.641 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[29491]: osdmap e74: 8 total, 8 up, 8 in 2024-09-17T08:40:25.641 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[29491]: mgrmap e29: y(active, starting, since 0.0054385s) 2024-09-17T08:40:25.641 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[29491]: from='mgr.? 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-17T08:40:25.641 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[29491]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-17T08:40:25.641 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[29491]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:40:25.641 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[29491]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:40:25.641 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[29491]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:40:25.641 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[29491]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-17T08:40:25.641 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[29491]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:40:25.641 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[29491]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:40:25.641 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[29491]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:40:25.641 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[29491]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:40:25.641 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[29491]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:40:25.641 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[29491]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:40:25.642 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[29491]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:40:25.642 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[29491]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:40:25.642 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[29491]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-17T08:40:25.642 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[29491]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-17T08:40:25.642 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[29491]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-17T08:40:25.642 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[29491]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T08:40:25.642 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[29491]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-17T08:40:25.642 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[29491]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T08:40:25.642 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: 2024-09-17T08:40:25.393+0000 7f5d8294a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T08:40:25.643 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: [17/Sep/2024:08:40:25] ENGINE Bus STARTING 2024-09-17T08:40:25.643 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: CherryPy Checker: 2024-09-17T08:40:25.643 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: The Application mounted at '' has an empty config. 2024-09-17T08:40:25.643 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: 2024-09-17T08:40:25.643 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: [17/Sep/2024:08:40:25] ENGINE Serving on http://:::9283 2024-09-17T08:40:25.643 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: [17/Sep/2024:08:40:25] ENGINE Bus STARTED 2024-09-17T08:40:25.643 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: [17/Sep/2024:08:40:25] ENGINE Bus STOPPING 2024-09-17T08:40:25.643 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: [17/Sep/2024:08:40:25] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-17T08:40:25.643 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: [17/Sep/2024:08:40:25] ENGINE Bus STOPPED 2024-09-17T08:40:25.643 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[41418]: from='client.? 172.21.15.118:0/3319445639' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-17T08:40:25.643 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[41418]: Active manager daemon y restarted 2024-09-17T08:40:25.643 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[41418]: Activating manager daemon y 2024-09-17T08:40:25.644 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[41418]: osdmap e74: 8 total, 8 up, 8 in 2024-09-17T08:40:25.644 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[41418]: mgrmap e29: y(active, starting, since 0.0054385s) 2024-09-17T08:40:25.644 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[41418]: from='mgr.? 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-17T08:40:25.644 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[41418]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-17T08:40:25.644 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[41418]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:40:25.644 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[41418]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:40:25.644 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[41418]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:40:25.644 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[41418]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-17T08:40:25.644 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[41418]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:40:25.644 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[41418]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:40:25.644 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[41418]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:40:25.644 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[41418]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:40:25.644 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[41418]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:40:25.644 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[41418]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:40:25.645 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[41418]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:40:25.645 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[41418]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:40:25.645 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[41418]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-17T08:40:25.645 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[41418]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-17T08:40:25.645 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[41418]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-17T08:40:25.645 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[41418]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T08:40:25.645 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[41418]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-17T08:40:25.645 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-mon[41418]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T08:40:25.881 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:25 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:25.599+0000 7fce9147c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:40:25.881 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:25 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:25.662+0000 7fce9147c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T08:40:25.881 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:25 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:25.811+0000 7fce9147c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T08:40:25.881 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:25 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:25.880+0000 7fce9147c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T08:40:25.899 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: [17/Sep/2024:08:40:25] ENGINE Bus STARTING 2024-09-17T08:40:25.899 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: [17/Sep/2024:08:40:25] ENGINE Serving on http://:::9283 2024-09-17T08:40:25.899 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:25 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: [17/Sep/2024:08:40:25] ENGINE Bus STARTED 2024-09-17T08:40:26.145 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:25 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:25.943+0000 7fce9147c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T08:40:26.145 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:26 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:26.106+0000 7fce9147c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T08:40:26.158 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-17T08:40:26.167 INFO:tasks.cephadm:Teardown begin 2024-09-17T08:40:26.168 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-17T08:40:26.201 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-17T08:40:26.232 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-17T08:40:26.232 DEBUG:teuthology.orchestra.run.smithi118:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-17T08:40:26.257 DEBUG:teuthology.orchestra.run.smithi170:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-17T08:40:26.289 INFO:tasks.cephadm:Stopping all daemons... 2024-09-17T08:40:26.289 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-17T08:40:26.289 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ceph-7862d192-74ce-11ef-bceb-c7b262605968@mon.a 2024-09-17T08:40:26.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:26 smithi118 systemd[1]: Stopping Ceph mon.a for 7862d192-74ce-11ef-bceb-c7b262605968... 2024-09-17T08:40:26.469 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:26 smithi170 ceph-mon[33271]: Manager daemon y is now available 2024-09-17T08:40:26.470 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:26 smithi170 ceph-mon[33271]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:40:26.470 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:26 smithi170 ceph-mon[33271]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:40:26.470 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:26 smithi170 ceph-mon[33271]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-17T08:40:26.470 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:26 smithi170 ceph-mon[33271]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-17T08:40:26.470 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:26 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:26.183+0000 7fce9147c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T08:40:26.470 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:26 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:26.258+0000 7fce9147c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T08:40:26.470 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:26 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:26.347+0000 7fce9147c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T08:40:26.470 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:26 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:26.406+0000 7fce9147c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T08:40:26.471 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:26 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:26.469+0000 7fce9147c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T08:40:26.687 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:26 smithi118 ceph-mon[29491]: Manager daemon y is now available 2024-09-17T08:40:26.687 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:26 smithi118 ceph-mon[29491]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:40:26.687 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:26 smithi118 ceph-mon[29491]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:40:26.687 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:26 smithi118 ceph-mon[29491]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-17T08:40:26.687 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:26 smithi118 ceph-mon[29491]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-17T08:40:26.688 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:26 smithi118 ceph-mon[41418]: Manager daemon y is now available 2024-09-17T08:40:26.688 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:26 smithi118 ceph-mon[41418]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:40:26.688 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:26 smithi118 ceph-mon[41418]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:40:26.688 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:26 smithi118 ceph-mon[41418]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-17T08:40:26.688 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:26 smithi118 ceph-mon[41418]: from='mgr.15402 172.21.15.118:0/3804562392' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-17T08:40:27.027 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:40:26 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a[79253]: ts=2024-09-17T08:40:26.914Z caller=notify.go:743 level=info component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify success" attempts=6 2024-09-17T08:40:27.027 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:26 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: 2024-09-17T08:40:26.915+0000 7f5cdad7d640 -1 log_channel(cephadm) log [ERR] : [17/Sep/2024:08:40:26] ENGINE Error in HTTPServer.serve 2024-09-17T08:40:27.028 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:26 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: Traceback (most recent call last): 2024-09-17T08:40:27.028 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:26 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T08:40:27.028 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:26 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: self._connections.run(self.expiration_interval) 2024-09-17T08:40:27.028 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:26 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T08:40:27.028 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:26 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: self._run(expiration_interval) 2024-09-17T08:40:27.028 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:26 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T08:40:27.028 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:26 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T08:40:27.028 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:26 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T08:40:27.028 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:26 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T08:40:27.028 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:26 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T08:40:27.028 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:26 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: s = self.context.wrap_socket( 2024-09-17T08:40:27.028 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:26 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T08:40:27.028 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:26 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: return self.sslsocket_class._create( 2024-09-17T08:40:27.029 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:26 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T08:40:27.029 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:26 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: self.do_handshake() 2024-09-17T08:40:27.029 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:26 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T08:40:27.029 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:26 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: self._sslobj.do_handshake() 2024-09-17T08:40:27.029 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:26 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T08:40:27.029 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:26 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: 2024-09-17T08:40:27.475 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:27 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mon-a[29468]: 2024-09-17T08:40:27.159+0000 7fd8c4d48640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:40:27.476 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:27 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mon-a[29468]: 2024-09-17T08:40:27.159+0000 7fd8c4d48640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-17T08:40:27.476 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:27 smithi118 podman[87007]: 2024-09-17 08:40:27.220960284 +0000 UTC m=+0.167425647 container died f167927aab3b4be72e05bb035c2d678ab85691374dc81d4a9bfd58cdc6f04b26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mon-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2) 2024-09-17T08:40:27.476 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:27 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: [17/Sep/2024:08:40:27] ENGINE Bus STOPPING 2024-09-17T08:40:27.476 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:27 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: ::ffff:172.21.15.170 - - [17/Sep/2024:08:40:27] "GET /metrics HTTP/1.1" 503 754 "" "Prometheus/2.43.0" 2024-09-17T08:40:27.476 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:27 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: [17/Sep/2024:08:40:27] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-17T08:40:27.476 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:27 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: [17/Sep/2024:08:40:27] ENGINE Bus STOPPED 2024-09-17T08:40:27.477 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:27 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: [17/Sep/2024:08:40:27] ENGINE Bus STARTING 2024-09-17T08:40:27.477 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:27 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: [17/Sep/2024:08:40:27] ENGINE Serving on http://:::9283 2024-09-17T08:40:27.477 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:27 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: [17/Sep/2024:08:40:27] ENGINE Bus STARTED 2024-09-17T08:40:27.620 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:27 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:27.291+0000 7fce9147c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T08:40:27.620 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:27 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: [17/Sep/2024:08:40:27] ENGINE Bus STARTING 2024-09-17T08:40:27.621 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:27 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: CherryPy Checker: 2024-09-17T08:40:27.621 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:27 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: The Application mounted at '' has an empty config. 2024-09-17T08:40:27.621 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:27 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: 2024-09-17T08:40:27.621 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:27 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: [17/Sep/2024:08:40:27] ENGINE Serving on http://:::9283 2024-09-17T08:40:27.621 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:27 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x[35351]: [17/Sep/2024:08:40:27] ENGINE Bus STARTED 2024-09-17T08:40:27.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:27 smithi118 podman[87007]: 2024-09-17 08:40:27.574790989 +0000 UTC m=+0.521256334 container cleanup f167927aab3b4be72e05bb035c2d678ab85691374dc81d4a9bfd58cdc6f04b26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mon-a, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T08:40:27.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:27 smithi118 bash[87007]: ceph-7862d192-74ce-11ef-bceb-c7b262605968-mon-a 2024-09-17T08:40:28.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Sep 17 08:40:27 smithi118 podman[87076]: 2024-09-17 08:40:27.892931102 +0000 UTC m=+0.666081151 container remove f167927aab3b4be72e05bb035c2d678ab85691374dc81d4a9bfd58cdc6f04b26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-17T08:40:28.424 DEBUG:teuthology.orchestra.run.smithi118:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7862d192-74ce-11ef-bceb-c7b262605968@mon.a.service' 2024-09-17T08:40:28.492 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:40:28.492 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-17T08:40:28.493 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-09-17T08:40:28.493 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ceph-7862d192-74ce-11ef-bceb-c7b262605968@mon.c 2024-09-17T08:40:28.777 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:28 smithi118 systemd[1]: Stopping Ceph mon.c for 7862d192-74ce-11ef-bceb-c7b262605968... 2024-09-17T08:40:28.895 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:40:28 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug there is no tcmu-runner data available 2024-09-17T08:40:30.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:29 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mon-c[41395]: 2024-09-17T08:40:29.696+0000 7faa65820640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:40:30.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:29 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mon-c[41395]: 2024-09-17T08:40:29.696+0000 7faa65820640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-09-17T08:40:30.027 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:29 smithi118 podman[87417]: 2024-09-17 08:40:29.748464625 +0000 UTC m=+0.504471022 container died 4cd40efe8dc5fe4f1d8a866d0890747eb0a3776890e93ca540d9ba1a40f2f297 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2) 2024-09-17T08:40:30.028 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:29 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: [17/Sep/2024:08:40:29] ENGINE Bus STOPPING 2024-09-17T08:40:30.028 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:29 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: [17/Sep/2024:08:40:29] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-17T08:40:30.028 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:29 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: [17/Sep/2024:08:40:29] ENGINE Bus STOPPED 2024-09-17T08:40:30.028 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:29 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: [17/Sep/2024:08:40:29] ENGINE Bus STARTING 2024-09-17T08:40:30.028 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:29 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: [17/Sep/2024:08:40:29] ENGINE Serving on http://:::9283 2024-09-17T08:40:30.028 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:29 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: [17/Sep/2024:08:40:29] ENGINE Bus STARTED 2024-09-17T08:40:30.396 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:30 smithi118 podman[87417]: 2024-09-17 08:40:30.153304222 +0000 UTC m=+0.909310574 container cleanup 4cd40efe8dc5fe4f1d8a866d0890747eb0a3776890e93ca540d9ba1a40f2f297 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mon-c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux ) 2024-09-17T08:40:30.396 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 17 08:40:30 smithi118 bash[87417]: ceph-7862d192-74ce-11ef-bceb-c7b262605968-mon-c 2024-09-17T08:40:30.568 DEBUG:teuthology.orchestra.run.smithi118:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7862d192-74ce-11ef-bceb-c7b262605968@mon.c.service' 2024-09-17T08:40:30.605 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:40:30.605 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-09-17T08:40:30.605 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-09-17T08:40:30.606 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-7862d192-74ce-11ef-bceb-c7b262605968@mon.b 2024-09-17T08:40:30.895 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:30 smithi170 systemd[1]: Stopping Ceph mon.b for 7862d192-74ce-11ef-bceb-c7b262605968... 2024-09-17T08:40:31.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:30 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mon-b[33248]: 2024-09-17T08:40:30.931+0000 7fb35038d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:40:31.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:30 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mon-b[33248]: 2024-09-17T08:40:30.931+0000 7fb35038d640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-09-17T08:40:31.395 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:31 smithi170 podman[73019]: 2024-09-17 08:40:31.167020965 +0000 UTC m=+0.368728908 container died c92809c44401538d290e476e6641c8490b95a3ee7bfe8079bc4ce2dbd5346c4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mon-b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=reef-19e2a2b, GIT_BRANCH=HEAD) 2024-09-17T08:40:31.813 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:31 smithi170 podman[73019]: 2024-09-17 08:40:31.579149371 +0000 UTC m=+0.780857308 container cleanup c92809c44401538d290e476e6641c8490b95a3ee7bfe8079bc4ce2dbd5346c4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mon-b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T08:40:31.813 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:31 smithi170 bash[73019]: ceph-7862d192-74ce-11ef-bceb-c7b262605968-mon-b 2024-09-17T08:40:32.058 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7862d192-74ce-11ef-bceb-c7b262605968@mon.b.service' 2024-09-17T08:40:32.080 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:31 smithi170 podman[73038]: 2024-09-17 08:40:31.813911429 +0000 UTC m=+0.640954332 container remove c92809c44401538d290e476e6641c8490b95a3ee7bfe8079bc4ce2dbd5346c4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mon-b, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-17T08:40:32.080 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:32 smithi170 systemd[1]: ceph-7862d192-74ce-11ef-bceb-c7b262605968@mon.b.service: Deactivated successfully. 2024-09-17T08:40:32.081 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:32 smithi170 systemd[1]: Stopped Ceph mon.b for 7862d192-74ce-11ef-bceb-c7b262605968. 2024-09-17T08:40:32.081 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 17 08:40:32 smithi170 systemd[1]: ceph-7862d192-74ce-11ef-bceb-c7b262605968@mon.b.service: Consumed 8.016s CPU time. 2024-09-17T08:40:32.104 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:40:32.104 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-09-17T08:40:32.104 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-17T08:40:32.104 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ceph-7862d192-74ce-11ef-bceb-c7b262605968@mgr.y 2024-09-17T08:40:32.409 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:32 smithi118 systemd[1]: Stopping Ceph mgr.y for 7862d192-74ce-11ef-bceb-c7b262605968... 2024-09-17T08:40:32.733 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:32 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: 2024-09-17T08:40:32.408+0000 7f5d7e8de640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:40:32.733 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:32 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y[86421]: 2024-09-17T08:40:32.408+0000 7f5d7e8de640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-17T08:40:32.734 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:32 smithi118 podman[87561]: 2024-09-17 08:40:32.441150435 +0000 UTC m=+0.140116363 container died ce4a9a4bb346d095e63fb3ca52089c1a12e39f92cacf35fa0c2f40720d720636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, RELEASE=reef-19e2a2b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD) 2024-09-17T08:40:33.027 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:32 smithi118 podman[87561]: 2024-09-17 08:40:32.815960876 +0000 UTC m=+0.514926804 container cleanup ce4a9a4bb346d095e63fb3ca52089c1a12e39f92cacf35fa0c2f40720d720636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-17T08:40:33.028 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:32 smithi118 bash[87561]: ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y 2024-09-17T08:40:33.261 DEBUG:teuthology.orchestra.run.smithi118:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7862d192-74ce-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-17T08:40:33.282 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:33 smithi118 podman[87577]: 2024-09-17 08:40:33.125501106 +0000 UTC m=+0.681484345 container remove ce4a9a4bb346d095e63fb3ca52089c1a12e39f92cacf35fa0c2f40720d720636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-y, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-17T08:40:33.282 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:33 smithi118 systemd[1]: ceph-7862d192-74ce-11ef-bceb-c7b262605968@mgr.y.service: Deactivated successfully. 2024-09-17T08:40:33.283 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:33 smithi118 systemd[1]: Stopped Ceph mgr.y for 7862d192-74ce-11ef-bceb-c7b262605968. 2024-09-17T08:40:33.283 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Sep 17 08:40:33 smithi118 systemd[1]: ceph-7862d192-74ce-11ef-bceb-c7b262605968@mgr.y.service: Consumed 7.531s CPU time. 2024-09-17T08:40:33.296 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:40:33.296 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-17T08:40:33.296 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-17T08:40:33.296 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-7862d192-74ce-11ef-bceb-c7b262605968@mgr.x 2024-09-17T08:40:33.625 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:33 smithi170 systemd[1]: Stopping Ceph mgr.x for 7862d192-74ce-11ef-bceb-c7b262605968... 2024-09-17T08:40:33.895 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:33 smithi170 podman[73150]: 2024-09-17 08:40:33.625046457 +0000 UTC m=+0.139068173 container died 2bd244aa4628633b716c2ea1d6347fcec992065db1b0958c2fa2761efe66bcf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T08:40:34.201 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:34 smithi170 podman[73150]: 2024-09-17 08:40:34.025239101 +0000 UTC m=+0.539260817 container cleanup 2bd244aa4628633b716c2ea1d6347fcec992065db1b0958c2fa2761efe66bcf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=reef-19e2a2b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-17T08:40:34.201 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:34 smithi170 bash[73150]: ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x 2024-09-17T08:40:34.451 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:34 smithi170 podman[73167]: 2024-09-17 08:40:34.201724045 +0000 UTC m=+0.576246183 container remove 2bd244aa4628633b716c2ea1d6347fcec992065db1b0958c2fa2761efe66bcf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-17T08:40:34.452 INFO:journalctl@ceph.mgr.x.smithi170.stdout:Sep 17 08:40:34 smithi170 systemd[1]: ceph-7862d192-74ce-11ef-bceb-c7b262605968@mgr.x.service: Main process exited, code=exited, status=143/n/a 2024-09-17T08:40:34.455 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7862d192-74ce-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-17T08:40:34.489 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:40:34.490 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-17T08:40:34.490 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-17T08:40:34.490 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ceph-7862d192-74ce-11ef-bceb-c7b262605968@osd.0 2024-09-17T08:40:34.614 INFO:journalctl@ceph.osd.0.smithi118.stdout:Sep 17 08:40:34 smithi118 systemd[1]: Stopping Ceph osd.0 for 7862d192-74ce-11ef-bceb-c7b262605968... 2024-09-17T08:40:35.028 INFO:journalctl@ceph.osd.0.smithi118.stdout:Sep 17 08:40:34 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-0[45818]: 2024-09-17T08:40:34.805+0000 7fd4903dd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:40:35.028 INFO:journalctl@ceph.osd.0.smithi118.stdout:Sep 17 08:40:34 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-0[45818]: 2024-09-17T08:40:34.805+0000 7fd4903dd640 -1 osd.0 74 *** Got signal Terminated *** 2024-09-17T08:40:35.028 INFO:journalctl@ceph.osd.0.smithi118.stdout:Sep 17 08:40:34 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-0[45818]: 2024-09-17T08:40:34.805+0000 7fd4903dd640 -1 osd.0 74 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T08:40:40.528 INFO:journalctl@ceph.osd.0.smithi118.stdout:Sep 17 08:40:40 smithi118 podman[87690]: 2024-09-17 08:40:40.241071193 +0000 UTC m=+5.560111341 container died 0f1b7bb92d0eb4703adbc9efc493575f0c742e29d90fef78a0bf334071d9e54e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-19e2a2b, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-17T08:40:40.845 INFO:journalctl@ceph.osd.0.smithi118.stdout:Sep 17 08:40:40 smithi118 podman[87690]: 2024-09-17 08:40:40.636391248 +0000 UTC m=+5.955431384 container cleanup 0f1b7bb92d0eb4703adbc9efc493575f0c742e29d90fef78a0bf334071d9e54e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b) 2024-09-17T08:40:40.846 INFO:journalctl@ceph.osd.0.smithi118.stdout:Sep 17 08:40:40 smithi118 bash[87690]: ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-0 2024-09-17T08:40:41.246 INFO:journalctl@ceph.osd.0.smithi118.stdout:Sep 17 08:40:40 smithi118 podman[87721]: 2024-09-17 08:40:40.846166874 +0000 UTC m=+0.602401601 container remove 0f1b7bb92d0eb4703adbc9efc493575f0c742e29d90fef78a0bf334071d9e54e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-0, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-17T08:40:41.527 INFO:journalctl@ceph.osd.0.smithi118.stdout:Sep 17 08:40:41 smithi118 podman[87816]: 2024-09-17 08:40:41.149407651 +0000 UTC m=+0.016064827 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T08:40:41.527 INFO:journalctl@ceph.osd.0.smithi118.stdout:Sep 17 08:40:41 smithi118 podman[87816]: 2024-09-17 08:40:41.339161851 +0000 UTC m=+0.205819051 container create e55591a8391c7bd2810b4f6efef4ae5ce97eb4e86253d54e33a6c3f88e894fff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-17T08:40:42.027 INFO:journalctl@ceph.osd.0.smithi118.stdout:Sep 17 08:40:41 smithi118 podman[87816]: 2024-09-17 08:40:41.583133198 +0000 UTC m=+0.449790368 container init e55591a8391c7bd2810b4f6efef4ae5ce97eb4e86253d54e33a6c3f88e894fff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_CLEAN=True, RELEASE=reef-19e2a2b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-17T08:40:42.027 INFO:journalctl@ceph.osd.0.smithi118.stdout:Sep 17 08:40:41 smithi118 podman[87816]: 2024-09-17 08:40:41.588694572 +0000 UTC m=+0.455351738 container start e55591a8391c7bd2810b4f6efef4ae5ce97eb4e86253d54e33a6c3f88e894fff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-19e2a2b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-17T08:40:42.027 INFO:journalctl@ceph.osd.0.smithi118.stdout:Sep 17 08:40:41 smithi118 podman[87816]: 2024-09-17 08:40:41.682933651 +0000 UTC m=+0.549590813 container attach e55591a8391c7bd2810b4f6efef4ae5ce97eb4e86253d54e33a6c3f88e894fff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-0-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=reef-19e2a2b, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-09-17T08:40:42.027 INFO:journalctl@ceph.osd.0.smithi118.stdout:Sep 17 08:40:41 smithi118 podman[87816]: 2024-09-17 08:40:41.772613482 +0000 UTC m=+0.639270651 container died e55591a8391c7bd2810b4f6efef4ae5ce97eb4e86253d54e33a6c3f88e894fff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b, ceph=True) 2024-09-17T08:40:42.452 DEBUG:teuthology.orchestra.run.smithi118:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7862d192-74ce-11ef-bceb-c7b262605968@osd.0.service' 2024-09-17T08:40:42.499 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:40:42.499 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-17T08:40:42.499 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-17T08:40:42.499 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ceph-7862d192-74ce-11ef-bceb-c7b262605968@osd.1 2024-09-17T08:40:42.777 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 17 08:40:42 smithi118 systemd[1]: Stopping Ceph osd.1 for 7862d192-74ce-11ef-bceb-c7b262605968... 2024-09-17T08:40:43.277 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 17 08:40:42 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-1[51742]: 2024-09-17T08:40:42.824+0000 7f0adfd11640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:40:43.277 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 17 08:40:42 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-1[51742]: 2024-09-17T08:40:42.824+0000 7f0adfd11640 -1 osd.1 74 *** Got signal Terminated *** 2024-09-17T08:40:43.277 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 17 08:40:42 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-1[51742]: 2024-09-17T08:40:42.824+0000 7f0adfd11640 -1 osd.1 74 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T08:40:48.632 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 17 08:40:48 smithi118 podman[87945]: 2024-09-17 08:40:48.346467944 +0000 UTC m=+5.667242260 container died 964613c9ee518b5d959351927bdf3cd29f83d9177ada8d1570d2df29a45de566 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-1, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, RELEASE=reef-19e2a2b) 2024-09-17T08:40:48.997 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 17 08:40:48 smithi118 podman[87945]: 2024-09-17 08:40:48.713889736 +0000 UTC m=+6.034664032 container cleanup 964613c9ee518b5d959351927bdf3cd29f83d9177ada8d1570d2df29a45de566 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-1, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True) 2024-09-17T08:40:48.998 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 17 08:40:48 smithi118 bash[87945]: ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-1 2024-09-17T08:40:49.277 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 17 08:40:48 smithi118 podman[87966]: 2024-09-17 08:40:48.998242226 +0000 UTC m=+0.653203873 container remove 964613c9ee518b5d959351927bdf3cd29f83d9177ada8d1570d2df29a45de566 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-17T08:40:49.580 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 17 08:40:49 smithi118 podman[88059]: 2024-09-17 08:40:49.231655531 +0000 UTC m=+0.016006262 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T08:40:49.580 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 17 08:40:49 smithi118 podman[88059]: 2024-09-17 08:40:49.365760374 +0000 UTC m=+0.150111101 container create 98d15c8b7e721f52fd4d572a165a2afef577e07b6a480476996890a2c1e87a58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-17T08:40:49.580 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 17 08:40:49 smithi118 podman[88059]: 2024-09-17 08:40:49.576452724 +0000 UTC m=+0.360803457 container init 98d15c8b7e721f52fd4d572a165a2afef577e07b6a480476996890a2c1e87a58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=reef-19e2a2b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-17T08:40:49.580 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 17 08:40:49 smithi118 podman[88059]: 2024-09-17 08:40:49.580873955 +0000 UTC m=+0.365224682 container start 98d15c8b7e721f52fd4d572a165a2afef577e07b6a480476996890a2c1e87a58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-17T08:40:50.027 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 17 08:40:49 smithi118 podman[88059]: 2024-09-17 08:40:49.684716713 +0000 UTC m=+0.469067463 container attach 98d15c8b7e721f52fd4d572a165a2afef577e07b6a480476996890a2c1e87a58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-1-deactivate, RELEASE=reef-19e2a2b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-17T08:40:50.027 INFO:journalctl@ceph.osd.1.smithi118.stdout:Sep 17 08:40:49 smithi118 podman[88059]: 2024-09-17 08:40:49.766222442 +0000 UTC m=+0.550573168 container died 98d15c8b7e721f52fd4d572a165a2afef577e07b6a480476996890a2c1e87a58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=reef-19e2a2b, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T08:40:50.433 DEBUG:teuthology.orchestra.run.smithi118:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7862d192-74ce-11ef-bceb-c7b262605968@osd.1.service' 2024-09-17T08:40:50.473 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:40:50.473 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-17T08:40:50.474 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-17T08:40:50.474 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ceph-7862d192-74ce-11ef-bceb-c7b262605968@osd.2 2024-09-17T08:40:50.777 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 17 08:40:50 smithi118 systemd[1]: Stopping Ceph osd.2 for 7862d192-74ce-11ef-bceb-c7b262605968... 2024-09-17T08:40:51.277 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 17 08:40:50 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-2[57824]: 2024-09-17T08:40:50.825+0000 7f39cc43a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:40:51.277 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 17 08:40:50 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-2[57824]: 2024-09-17T08:40:50.825+0000 7f39cc43a640 -1 osd.2 74 *** Got signal Terminated *** 2024-09-17T08:40:51.277 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 17 08:40:50 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-2[57824]: 2024-09-17T08:40:50.825+0000 7f39cc43a640 -1 osd.2 74 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T08:40:56.600 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 17 08:40:56 smithi118 podman[88189]: 2024-09-17 08:40:56.339570959 +0000 UTC m=+5.632859426 container died 21947b47c74b9639df06b42fbaa8c7997d4711e086656216f2a2d96bba2ec0ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=reef-19e2a2b) 2024-09-17T08:40:56.883 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 17 08:40:56 smithi118 podman[88189]: 2024-09-17 08:40:56.707152724 +0000 UTC m=+6.000441174 container cleanup 21947b47c74b9639df06b42fbaa8c7997d4711e086656216f2a2d96bba2ec0ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T08:40:56.883 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 17 08:40:56 smithi118 bash[88189]: ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-2 2024-09-17T08:40:57.277 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 17 08:40:56 smithi118 podman[88209]: 2024-09-17 08:40:56.883248998 +0000 UTC m=+0.536409599 container remove 21947b47c74b9639df06b42fbaa8c7997d4711e086656216f2a2d96bba2ec0ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, org.label-schema.vendor=CentOS) 2024-09-17T08:40:57.661 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 17 08:40:57 smithi118 podman[88304]: 2024-09-17 08:40:57.220461086 +0000 UTC m=+0.016144150 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T08:40:57.661 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 17 08:40:57 smithi118 podman[88304]: 2024-09-17 08:40:57.342417708 +0000 UTC m=+0.138100802 container create 9155b26b752a1e8c439ff0e0a30a1034e06576d362b7a6159aa99ba33e036adc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-2-deactivate, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-17T08:40:57.661 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 17 08:40:57 smithi118 podman[88304]: 2024-09-17 08:40:57.519780492 +0000 UTC m=+0.315463551 container init 9155b26b752a1e8c439ff0e0a30a1034e06576d362b7a6159aa99ba33e036adc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-17T08:40:57.661 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 17 08:40:57 smithi118 podman[88304]: 2024-09-17 08:40:57.524708902 +0000 UTC m=+0.320391952 container start 9155b26b752a1e8c439ff0e0a30a1034e06576d362b7a6159aa99ba33e036adc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-2-deactivate, RELEASE=reef-19e2a2b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-17T08:40:57.661 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 17 08:40:57 smithi118 podman[88304]: 2024-09-17 08:40:57.661308427 +0000 UTC m=+0.456991478 container attach 9155b26b752a1e8c439ff0e0a30a1034e06576d362b7a6159aa99ba33e036adc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T08:40:58.027 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 17 08:40:57 smithi118 podman[88304]: 2024-09-17 08:40:57.717025683 +0000 UTC m=+0.512708733 container died 9155b26b752a1e8c439ff0e0a30a1034e06576d362b7a6159aa99ba33e036adc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T08:40:58.347 DEBUG:teuthology.orchestra.run.smithi118:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7862d192-74ce-11ef-bceb-c7b262605968@osd.2.service' 2024-09-17T08:40:58.389 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:40:58.389 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-17T08:40:58.389 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-17T08:40:58.389 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ceph-7862d192-74ce-11ef-bceb-c7b262605968@osd.3 2024-09-17T08:40:58.710 INFO:journalctl@ceph.osd.3.smithi118.stdout:Sep 17 08:40:58 smithi118 systemd[1]: Stopping Ceph osd.3 for 7862d192-74ce-11ef-bceb-c7b262605968... 2024-09-17T08:40:59.027 INFO:journalctl@ceph.osd.3.smithi118.stdout:Sep 17 08:40:58 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-3[64385]: 2024-09-17T08:40:58.710+0000 7fb52054e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:40:59.027 INFO:journalctl@ceph.osd.3.smithi118.stdout:Sep 17 08:40:58 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-3[64385]: 2024-09-17T08:40:58.710+0000 7fb52054e640 -1 osd.3 74 *** Got signal Terminated *** 2024-09-17T08:40:59.027 INFO:journalctl@ceph.osd.3.smithi118.stdout:Sep 17 08:40:58 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-3[64385]: 2024-09-17T08:40:58.710+0000 7fb52054e640 -1 osd.3 74 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T08:41:01.777 INFO:journalctl@ceph.osd.3.smithi118.stdout:Sep 17 08:41:01 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-3[64385]: 2024-09-17T08:41:01.340+0000 7fb518b5a640 -1 osd.3 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:35.579415+0000 front 2024-09-17T08:40:35.579489+0000 (oldest deadline 2024-09-17T08:41:00.879470+0000) 2024-09-17T08:41:01.895 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 17 08:41:01 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-4[38685]: 2024-09-17T08:41:01.397+0000 7f60ef5b5640 -1 osd.4 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:36.648627+0000 front 2024-09-17T08:40:36.648569+0000 (oldest deadline 2024-09-17T08:41:01.348603+0000) 2024-09-17T08:41:02.160 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:01 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5[44539]: 2024-09-17T08:41:01.965+0000 7f21d8ede640 -1 osd.5 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:39.673962+0000 front 2024-09-17T08:40:39.674054+0000 (oldest deadline 2024-09-17T08:41:01.373856+0000) 2024-09-17T08:41:02.777 INFO:journalctl@ceph.osd.3.smithi118.stdout:Sep 17 08:41:02 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-3[64385]: 2024-09-17T08:41:02.385+0000 7fb518b5a640 -1 osd.3 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:35.579415+0000 front 2024-09-17T08:40:35.579489+0000 (oldest deadline 2024-09-17T08:41:00.879470+0000) 2024-09-17T08:41:02.842 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 17 08:41:02 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-4[38685]: 2024-09-17T08:41:02.417+0000 7f60ef5b5640 -1 osd.4 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:36.648627+0000 front 2024-09-17T08:40:36.648569+0000 (oldest deadline 2024-09-17T08:41:01.348603+0000) 2024-09-17T08:41:02.842 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:41:02 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:41:02.459Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.118:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.118:8765: connect: connection refused" 2024-09-17T08:41:02.843 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:41:02 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:41:02.461Z caller=refresh.go:99 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.118:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.118:8765: connect: connection refused" 2024-09-17T08:41:02.843 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:41:02 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:41:02.462Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.118:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.118:8765: connect: connection refused" 2024-09-17T08:41:02.843 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:41:02 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:41:02.463Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=nvmeof msg="Unable to refresh target groups" err="Get \"http://172.21.15.118:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.118:8765: connect: connection refused" 2024-09-17T08:41:02.843 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:41:02 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:41:02.463Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.118:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.118:8765: connect: connection refused" 2024-09-17T08:41:03.122 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:03 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5[44539]: 2024-09-17T08:41:03.006+0000 7f21d8ede640 -1 osd.5 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:39.673962+0000 front 2024-09-17T08:40:39.674054+0000 (oldest deadline 2024-09-17T08:41:01.373856+0000) 2024-09-17T08:41:03.122 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:02 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:02.841+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:35.995970+0000 front 2024-09-17T08:40:35.996185+0000 (oldest deadline 2024-09-17T08:41:01.896184+0000) 2024-09-17T08:41:03.678 INFO:journalctl@ceph.osd.3.smithi118.stdout:Sep 17 08:41:03 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-3[64385]: 2024-09-17T08:41:03.420+0000 7fb518b5a640 -1 osd.3 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:35.579415+0000 front 2024-09-17T08:40:35.579489+0000 (oldest deadline 2024-09-17T08:41:00.879470+0000) 2024-09-17T08:41:03.802 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:03 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:03.646+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:37.983615+0000 front 2024-09-17T08:40:37.983624+0000 (oldest deadline 2024-09-17T08:41:03.283463+0000) 2024-09-17T08:41:03.802 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 17 08:41:03 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-4[38685]: 2024-09-17T08:41:03.426+0000 7f60ef5b5640 -1 osd.4 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:36.648627+0000 front 2024-09-17T08:40:36.648569+0000 (oldest deadline 2024-09-17T08:41:01.348603+0000) 2024-09-17T08:41:04.145 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:04 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5[44539]: 2024-09-17T08:41:04.046+0000 7f21d8ede640 -1 osd.5 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:39.673962+0000 front 2024-09-17T08:40:39.674054+0000 (oldest deadline 2024-09-17T08:41:01.373856+0000) 2024-09-17T08:41:04.146 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:03 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:03.801+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:35.995970+0000 front 2024-09-17T08:40:35.996185+0000 (oldest deadline 2024-09-17T08:41:01.896184+0000) 2024-09-17T08:41:04.517 INFO:journalctl@ceph.osd.3.smithi118.stdout:Sep 17 08:41:04 smithi118 podman[88431]: 2024-09-17 08:41:04.243385958 +0000 UTC m=+5.634430126 container died ae6263c8777baa720f1ee4cc96cd82d45342f3632d167a33c36b698f9bda4ac7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-3, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-17T08:41:04.645 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:04 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:04.601+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:37.983615+0000 front 2024-09-17T08:40:37.983624+0000 (oldest deadline 2024-09-17T08:41:03.283463+0000) 2024-09-17T08:41:04.645 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 17 08:41:04 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-4[38685]: 2024-09-17T08:41:04.384+0000 7f60ef5b5640 -1 osd.4 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:36.648627+0000 front 2024-09-17T08:40:36.648569+0000 (oldest deadline 2024-09-17T08:41:01.348603+0000) 2024-09-17T08:41:04.777 INFO:journalctl@ceph.osd.3.smithi118.stdout:Sep 17 08:41:04 smithi118 podman[88431]: 2024-09-17 08:41:04.616175939 +0000 UTC m=+6.007220106 container cleanup ae6263c8777baa720f1ee4cc96cd82d45342f3632d167a33c36b698f9bda4ac7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-3, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T08:41:04.777 INFO:journalctl@ceph.osd.3.smithi118.stdout:Sep 17 08:41:04 smithi118 bash[88431]: ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-3 2024-09-17T08:41:05.064 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:04 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:04.760+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:35.995970+0000 front 2024-09-17T08:40:35.996185+0000 (oldest deadline 2024-09-17T08:41:01.896184+0000) 2024-09-17T08:41:05.227 INFO:journalctl@ceph.osd.3.smithi118.stdout:Sep 17 08:41:04 smithi118 podman[88460]: 2024-09-17 08:41:04.884341369 +0000 UTC m=+0.642092730 container remove ae6263c8777baa720f1ee4cc96cd82d45342f3632d167a33c36b698f9bda4ac7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-3, RELEASE=reef-19e2a2b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T08:41:05.385 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:05 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5[44539]: 2024-09-17T08:41:05.064+0000 7f21d8ede640 -1 osd.5 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:39.673962+0000 front 2024-09-17T08:40:39.674054+0000 (oldest deadline 2024-09-17T08:41:01.373856+0000) 2024-09-17T08:41:05.504 INFO:journalctl@ceph.osd.3.smithi118.stdout:Sep 17 08:41:05 smithi118 podman[88555]: 2024-09-17 08:41:05.13063231 +0000 UTC m=+0.015635459 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T08:41:05.504 INFO:journalctl@ceph.osd.3.smithi118.stdout:Sep 17 08:41:05 smithi118 podman[88555]: 2024-09-17 08:41:05.276950434 +0000 UTC m=+0.161953573 container create a43389c6313de5a0a6f62b3c50030254282011a8c70e63ad09680189cec68ace (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T08:41:05.504 INFO:journalctl@ceph.osd.3.smithi118.stdout:Sep 17 08:41:05 smithi118 podman[88555]: 2024-09-17 08:41:05.504479124 +0000 UTC m=+0.389482263 container init a43389c6313de5a0a6f62b3c50030254282011a8c70e63ad09680189cec68ace (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=reef-19e2a2b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T08:41:05.645 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 17 08:41:05 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-4[38685]: 2024-09-17T08:41:05.385+0000 7f60ef5b5640 -1 osd.4 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:36.648627+0000 front 2024-09-17T08:40:36.648569+0000 (oldest deadline 2024-09-17T08:41:01.348603+0000) 2024-09-17T08:41:05.645 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:05 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:05.631+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:37.983615+0000 front 2024-09-17T08:40:37.983624+0000 (oldest deadline 2024-09-17T08:41:03.283463+0000) 2024-09-17T08:41:05.777 INFO:journalctl@ceph.osd.3.smithi118.stdout:Sep 17 08:41:05 smithi118 podman[88555]: 2024-09-17 08:41:05.50914409 +0000 UTC m=+0.394147231 container start a43389c6313de5a0a6f62b3c50030254282011a8c70e63ad09680189cec68ace (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-17T08:41:05.777 INFO:journalctl@ceph.osd.3.smithi118.stdout:Sep 17 08:41:05 smithi118 podman[88555]: 2024-09-17 08:41:05.587423008 +0000 UTC m=+0.472426161 container attach a43389c6313de5a0a6f62b3c50030254282011a8c70e63ad09680189cec68ace (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-17T08:41:05.777 INFO:journalctl@ceph.osd.3.smithi118.stdout:Sep 17 08:41:05 smithi118 podman[88555]: 2024-09-17 08:41:05.703883527 +0000 UTC m=+0.588886676 container died a43389c6313de5a0a6f62b3c50030254282011a8c70e63ad09680189cec68ace (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, ceph=True, RELEASE=reef-19e2a2b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-17T08:41:06.085 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:05 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:05.755+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:35.995970+0000 front 2024-09-17T08:40:35.996185+0000 (oldest deadline 2024-09-17T08:41:01.896184+0000) 2024-09-17T08:41:06.281 DEBUG:teuthology.orchestra.run.smithi118:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7862d192-74ce-11ef-bceb-c7b262605968@osd.3.service' 2024-09-17T08:41:06.321 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:41:06.321 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-17T08:41:06.321 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-17T08:41:06.322 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-7862d192-74ce-11ef-bceb-c7b262605968@osd.4 2024-09-17T08:41:06.349 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:06 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5[44539]: 2024-09-17T08:41:06.085+0000 7f21d8ede640 -1 osd.5 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:39.673962+0000 front 2024-09-17T08:40:39.674054+0000 (oldest deadline 2024-09-17T08:41:01.373856+0000) 2024-09-17T08:41:06.645 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 17 08:41:06 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-4[38685]: 2024-09-17T08:41:06.349+0000 7f60ef5b5640 -1 osd.4 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:36.648627+0000 front 2024-09-17T08:40:36.648569+0000 (oldest deadline 2024-09-17T08:41:01.348603+0000) 2024-09-17T08:41:06.645 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 17 08:41:06 smithi170 systemd[1]: Stopping Ceph osd.4 for 7862d192-74ce-11ef-bceb-c7b262605968... 2024-09-17T08:41:07.082 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:06 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:06.657+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:37.983615+0000 front 2024-09-17T08:40:37.983624+0000 (oldest deadline 2024-09-17T08:41:03.283463+0000) 2024-09-17T08:41:07.082 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:06 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:06.740+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:35.995970+0000 front 2024-09-17T08:40:35.996185+0000 (oldest deadline 2024-09-17T08:41:01.896184+0000) 2024-09-17T08:41:07.082 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 17 08:41:06 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-4[38685]: 2024-09-17T08:41:06.645+0000 7f60f6fa9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:41:07.083 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 17 08:41:06 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-4[38685]: 2024-09-17T08:41:06.645+0000 7f60f6fa9640 -1 osd.4 74 *** Got signal Terminated *** 2024-09-17T08:41:07.083 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 17 08:41:06 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-4[38685]: 2024-09-17T08:41:06.645+0000 7f60f6fa9640 -1 osd.4 74 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T08:41:07.350 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:07 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5[44539]: 2024-09-17T08:41:07.081+0000 7f21d8ede640 -1 osd.5 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:39.673962+0000 front 2024-09-17T08:40:39.674054+0000 (oldest deadline 2024-09-17T08:41:01.373856+0000) 2024-09-17T08:41:07.628 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 17 08:41:07 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-4[38685]: 2024-09-17T08:41:07.350+0000 7f60ef5b5640 -1 osd.4 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:36.648627+0000 front 2024-09-17T08:40:36.648569+0000 (oldest deadline 2024-09-17T08:41:01.348603+0000) 2024-09-17T08:41:07.895 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:07 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:07.755+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:35.995970+0000 front 2024-09-17T08:40:35.996185+0000 (oldest deadline 2024-09-17T08:41:01.896184+0000) 2024-09-17T08:41:07.895 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:07 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:07.627+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:37.983615+0000 front 2024-09-17T08:40:37.983624+0000 (oldest deadline 2024-09-17T08:41:03.283463+0000) 2024-09-17T08:41:08.355 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:08 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5[44539]: 2024-09-17T08:41:08.038+0000 7f21d8ede640 -1 osd.5 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:39.673962+0000 front 2024-09-17T08:40:39.674054+0000 (oldest deadline 2024-09-17T08:41:01.373856+0000) 2024-09-17T08:41:08.645 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 17 08:41:08 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-4[38685]: 2024-09-17T08:41:08.355+0000 7f60ef5b5640 -1 osd.4 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:36.648627+0000 front 2024-09-17T08:40:36.648569+0000 (oldest deadline 2024-09-17T08:41:01.348603+0000) 2024-09-17T08:41:09.057 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:08 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:08.720+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:35.995970+0000 front 2024-09-17T08:40:35.996185+0000 (oldest deadline 2024-09-17T08:41:01.896184+0000) 2024-09-17T08:41:09.057 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:08 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:08.655+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:37.983615+0000 front 2024-09-17T08:40:37.983624+0000 (oldest deadline 2024-09-17T08:41:03.283463+0000) 2024-09-17T08:41:09.320 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:09 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5[44539]: 2024-09-17T08:41:09.056+0000 7f21d8ede640 -1 osd.5 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:39.673962+0000 front 2024-09-17T08:40:39.674054+0000 (oldest deadline 2024-09-17T08:41:01.373856+0000) 2024-09-17T08:41:09.617 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 17 08:41:09 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-4[38685]: 2024-09-17T08:41:09.319+0000 7f60ef5b5640 -1 osd.4 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:36.648627+0000 front 2024-09-17T08:40:36.648569+0000 (oldest deadline 2024-09-17T08:41:01.348603+0000) 2024-09-17T08:41:09.895 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:09 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:09.616+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:37.983615+0000 front 2024-09-17T08:40:37.983624+0000 (oldest deadline 2024-09-17T08:41:03.283463+0000) 2024-09-17T08:41:09.895 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:09 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:09.762+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:35.995970+0000 front 2024-09-17T08:40:35.996185+0000 (oldest deadline 2024-09-17T08:41:01.896184+0000) 2024-09-17T08:41:09.896 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:09 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:09.762+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:44.697871+0000 front 2024-09-17T08:40:44.697971+0000 (oldest deadline 2024-09-17T08:41:09.397883+0000) 2024-09-17T08:41:10.362 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:10 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5[44539]: 2024-09-17T08:41:10.036+0000 7f21d8ede640 -1 osd.5 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:39.673962+0000 front 2024-09-17T08:40:39.674054+0000 (oldest deadline 2024-09-17T08:41:01.373856+0000) 2024-09-17T08:41:10.636 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 17 08:41:10 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-4[38685]: 2024-09-17T08:41:10.361+0000 7f60ef5b5640 -1 osd.4 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:36.648627+0000 front 2024-09-17T08:40:36.648569+0000 (oldest deadline 2024-09-17T08:41:01.348603+0000) 2024-09-17T08:41:10.895 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:10 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:10.636+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:37.983615+0000 front 2024-09-17T08:40:37.983624+0000 (oldest deadline 2024-09-17T08:41:03.283463+0000) 2024-09-17T08:41:10.895 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:10 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:10.790+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:35.995970+0000 front 2024-09-17T08:40:35.996185+0000 (oldest deadline 2024-09-17T08:41:01.896184+0000) 2024-09-17T08:41:10.896 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:10 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:10.790+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:44.697871+0000 front 2024-09-17T08:40:44.697971+0000 (oldest deadline 2024-09-17T08:41:09.397883+0000) 2024-09-17T08:41:11.359 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:11 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5[44539]: 2024-09-17T08:41:11.032+0000 7f21d8ede640 -1 osd.5 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:39.673962+0000 front 2024-09-17T08:40:39.674054+0000 (oldest deadline 2024-09-17T08:41:01.373856+0000) 2024-09-17T08:41:11.359 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:11 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5[44539]: 2024-09-17T08:41:11.032+0000 7f21d8ede640 -1 osd.5 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:47.275114+0000 front 2024-09-17T08:40:47.275068+0000 (oldest deadline 2024-09-17T08:41:10.174959+0000) 2024-09-17T08:41:11.645 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:11 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:11.589+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:37.983615+0000 front 2024-09-17T08:40:37.983624+0000 (oldest deadline 2024-09-17T08:41:03.283463+0000) 2024-09-17T08:41:11.645 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 17 08:41:11 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-4[38685]: 2024-09-17T08:41:11.359+0000 7f60ef5b5640 -1 osd.4 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:36.648627+0000 front 2024-09-17T08:40:36.648569+0000 (oldest deadline 2024-09-17T08:41:01.348603+0000) 2024-09-17T08:41:11.646 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 17 08:41:11 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-4[38685]: 2024-09-17T08:41:11.359+0000 7f60ef5b5640 -1 osd.4 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:46.649402+0000 front 2024-09-17T08:40:46.649417+0000 (oldest deadline 2024-09-17T08:41:11.349468+0000) 2024-09-17T08:41:12.082 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 17 08:41:12 smithi170 podman[73296]: 2024-09-17 08:41:12.040121009 +0000 UTC m=+5.518813124 container died 4ca79f927caa6887124bb39ab031f745fe84ca29cfc7fd056e0fcee8a99c32d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T08:41:12.082 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:11 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:11.823+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:35.995970+0000 front 2024-09-17T08:40:35.996185+0000 (oldest deadline 2024-09-17T08:41:01.896184+0000) 2024-09-17T08:41:12.082 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:11 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:11.823+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:44.697871+0000 front 2024-09-17T08:40:44.697971+0000 (oldest deadline 2024-09-17T08:41:09.397883+0000) 2024-09-17T08:41:12.395 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:12 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5[44539]: 2024-09-17T08:41:12.081+0000 7f21d8ede640 -1 osd.5 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:39.673962+0000 front 2024-09-17T08:40:39.674054+0000 (oldest deadline 2024-09-17T08:41:01.373856+0000) 2024-09-17T08:41:12.395 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:12 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5[44539]: 2024-09-17T08:41:12.081+0000 7f21d8ede640 -1 osd.5 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:47.275114+0000 front 2024-09-17T08:40:47.275068+0000 (oldest deadline 2024-09-17T08:41:10.174959+0000) 2024-09-17T08:41:12.687 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:12 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:12.638+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:37.983615+0000 front 2024-09-17T08:40:37.983624+0000 (oldest deadline 2024-09-17T08:41:03.283463+0000) 2024-09-17T08:41:12.687 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 17 08:41:12 smithi170 podman[73296]: 2024-09-17 08:41:12.402134158 +0000 UTC m=+5.880826272 container cleanup 4ca79f927caa6887124bb39ab031f745fe84ca29cfc7fd056e0fcee8a99c32d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True) 2024-09-17T08:41:12.688 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 17 08:41:12 smithi170 bash[73296]: ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-4 2024-09-17T08:41:13.009 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:12 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:12.814+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:35.995970+0000 front 2024-09-17T08:40:35.996185+0000 (oldest deadline 2024-09-17T08:41:01.896184+0000) 2024-09-17T08:41:13.009 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:12 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:12.814+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:44.697871+0000 front 2024-09-17T08:40:44.697971+0000 (oldest deadline 2024-09-17T08:41:09.397883+0000) 2024-09-17T08:41:13.009 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 17 08:41:12 smithi170 podman[73316]: 2024-09-17 08:41:12.686963916 +0000 UTC m=+0.643371264 container remove 4ca79f927caa6887124bb39ab031f745fe84ca29cfc7fd056e0fcee8a99c32d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T08:41:13.265 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:13 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5[44539]: 2024-09-17T08:41:13.048+0000 7f21d8ede640 -1 osd.5 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:39.673962+0000 front 2024-09-17T08:40:39.674054+0000 (oldest deadline 2024-09-17T08:41:01.373856+0000) 2024-09-17T08:41:13.265 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:13 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5[44539]: 2024-09-17T08:41:13.048+0000 7f21d8ede640 -1 osd.5 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:47.275114+0000 front 2024-09-17T08:40:47.275068+0000 (oldest deadline 2024-09-17T08:41:10.174959+0000) 2024-09-17T08:41:13.266 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 17 08:41:13 smithi170 podman[73412]: 2024-09-17 08:41:12.912324229 +0000 UTC m=+0.016186262 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T08:41:13.266 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 17 08:41:13 smithi170 podman[73412]: 2024-09-17 08:41:13.054790173 +0000 UTC m=+0.158652237 container create f6be4e05db155d6f1d3734785df7080a6c474c73a282eaff066c3fded88c98df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, RELEASE=reef-19e2a2b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-17T08:41:13.266 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 17 08:41:13 smithi170 podman[73412]: 2024-09-17 08:41:13.265632716 +0000 UTC m=+0.369494746 container init f6be4e05db155d6f1d3734785df7080a6c474c73a282eaff066c3fded88c98df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-17T08:41:13.645 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 17 08:41:13 smithi170 podman[73412]: 2024-09-17 08:41:13.270845474 +0000 UTC m=+0.374707497 container start f6be4e05db155d6f1d3734785df7080a6c474c73a282eaff066c3fded88c98df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-17T08:41:13.645 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 17 08:41:13 smithi170 podman[73412]: 2024-09-17 08:41:13.357237567 +0000 UTC m=+0.461099597 container attach f6be4e05db155d6f1d3734785df7080a6c474c73a282eaff066c3fded88c98df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-17T08:41:13.646 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 17 08:41:13 smithi170 podman[73412]: 2024-09-17 08:41:13.455322074 +0000 UTC m=+0.559184107 container died f6be4e05db155d6f1d3734785df7080a6c474c73a282eaff066c3fded88c98df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, ceph=True, RELEASE=reef-19e2a2b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-17T08:41:14.052 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:13 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:13.649+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:37.983615+0000 front 2024-09-17T08:40:37.983624+0000 (oldest deadline 2024-09-17T08:41:03.283463+0000) 2024-09-17T08:41:14.053 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:13 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:13.847+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:35.995970+0000 front 2024-09-17T08:40:35.996185+0000 (oldest deadline 2024-09-17T08:41:01.896184+0000) 2024-09-17T08:41:14.053 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:13 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:13.847+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:44.697871+0000 front 2024-09-17T08:40:44.697971+0000 (oldest deadline 2024-09-17T08:41:09.397883+0000) 2024-09-17T08:41:14.070 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7862d192-74ce-11ef-bceb-c7b262605968@osd.4.service' 2024-09-17T08:41:14.150 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:41:14.150 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-17T08:41:14.150 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-17T08:41:14.151 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-7862d192-74ce-11ef-bceb-c7b262605968@osd.5 2024-09-17T08:41:14.395 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:14 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5[44539]: 2024-09-17T08:41:14.065+0000 7f21d8ede640 -1 osd.5 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:39.673962+0000 front 2024-09-17T08:40:39.674054+0000 (oldest deadline 2024-09-17T08:41:01.373856+0000) 2024-09-17T08:41:14.395 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:14 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5[44539]: 2024-09-17T08:41:14.065+0000 7f21d8ede640 -1 osd.5 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:47.275114+0000 front 2024-09-17T08:40:47.275068+0000 (oldest deadline 2024-09-17T08:41:10.174959+0000) 2024-09-17T08:41:14.395 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:14 smithi170 systemd[1]: Stopping Ceph osd.5 for 7862d192-74ce-11ef-bceb-c7b262605968... 2024-09-17T08:41:14.832 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:14 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:14.642+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:37.983615+0000 front 2024-09-17T08:40:37.983624+0000 (oldest deadline 2024-09-17T08:41:03.283463+0000) 2024-09-17T08:41:14.833 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:14 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:14.642+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:43.284051+0000 front 2024-09-17T08:40:43.284038+0000 (oldest deadline 2024-09-17T08:41:13.884456+0000) 2024-09-17T08:41:14.833 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:14 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5[44539]: 2024-09-17T08:41:14.507+0000 7f21dc8ca640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:41:14.833 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:14 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5[44539]: 2024-09-17T08:41:14.507+0000 7f21dc8ca640 -1 osd.5 74 *** Got signal Terminated *** 2024-09-17T08:41:14.833 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:14 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5[44539]: 2024-09-17T08:41:14.507+0000 7f21dc8ca640 -1 osd.5 74 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T08:41:15.111 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:14 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:14.832+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:35.995970+0000 front 2024-09-17T08:40:35.996185+0000 (oldest deadline 2024-09-17T08:41:01.896184+0000) 2024-09-17T08:41:15.111 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:14 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:14.832+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:44.697871+0000 front 2024-09-17T08:40:44.697971+0000 (oldest deadline 2024-09-17T08:41:09.397883+0000) 2024-09-17T08:41:15.395 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:15 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5[44539]: 2024-09-17T08:41:15.110+0000 7f21d8ede640 -1 osd.5 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:39.673962+0000 front 2024-09-17T08:40:39.674054+0000 (oldest deadline 2024-09-17T08:41:01.373856+0000) 2024-09-17T08:41:15.395 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:15 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5[44539]: 2024-09-17T08:41:15.110+0000 7f21d8ede640 -1 osd.5 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:47.275114+0000 front 2024-09-17T08:40:47.275068+0000 (oldest deadline 2024-09-17T08:41:10.174959+0000) 2024-09-17T08:41:16.141 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:15 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:15.652+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:37.983615+0000 front 2024-09-17T08:40:37.983624+0000 (oldest deadline 2024-09-17T08:41:03.283463+0000) 2024-09-17T08:41:16.141 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:15 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:15.652+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:43.284051+0000 front 2024-09-17T08:40:43.284038+0000 (oldest deadline 2024-09-17T08:41:13.884456+0000) 2024-09-17T08:41:16.141 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:15 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:15.831+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:35.995970+0000 front 2024-09-17T08:40:35.996185+0000 (oldest deadline 2024-09-17T08:41:01.896184+0000) 2024-09-17T08:41:16.141 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:15 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:15.831+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:44.697871+0000 front 2024-09-17T08:40:44.697971+0000 (oldest deadline 2024-09-17T08:41:09.397883+0000) 2024-09-17T08:41:16.395 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:16 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5[44539]: 2024-09-17T08:41:16.140+0000 7f21d8ede640 -1 osd.5 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:39.673962+0000 front 2024-09-17T08:40:39.674054+0000 (oldest deadline 2024-09-17T08:41:01.373856+0000) 2024-09-17T08:41:16.395 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:16 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5[44539]: 2024-09-17T08:41:16.140+0000 7f21d8ede640 -1 osd.5 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:47.275114+0000 front 2024-09-17T08:40:47.275068+0000 (oldest deadline 2024-09-17T08:41:10.174959+0000) 2024-09-17T08:41:17.106 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:16 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:16.874+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:35.995970+0000 front 2024-09-17T08:40:35.996185+0000 (oldest deadline 2024-09-17T08:41:01.896184+0000) 2024-09-17T08:41:17.106 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:16 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:16.874+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:44.697871+0000 front 2024-09-17T08:40:44.697971+0000 (oldest deadline 2024-09-17T08:41:09.397883+0000) 2024-09-17T08:41:17.107 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:16 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:16.682+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:37.983615+0000 front 2024-09-17T08:40:37.983624+0000 (oldest deadline 2024-09-17T08:41:03.283463+0000) 2024-09-17T08:41:17.107 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:16 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:16.682+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:43.284051+0000 front 2024-09-17T08:40:43.284038+0000 (oldest deadline 2024-09-17T08:41:13.884456+0000) 2024-09-17T08:41:17.395 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:17 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5[44539]: 2024-09-17T08:41:17.106+0000 7f21d8ede640 -1 osd.5 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:39.673962+0000 front 2024-09-17T08:40:39.674054+0000 (oldest deadline 2024-09-17T08:41:01.373856+0000) 2024-09-17T08:41:17.395 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:17 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5[44539]: 2024-09-17T08:41:17.106+0000 7f21d8ede640 -1 osd.5 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:47.275114+0000 front 2024-09-17T08:40:47.275068+0000 (oldest deadline 2024-09-17T08:41:10.174959+0000) 2024-09-17T08:41:18.064 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:17 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:17.716+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:37.983615+0000 front 2024-09-17T08:40:37.983624+0000 (oldest deadline 2024-09-17T08:41:03.283463+0000) 2024-09-17T08:41:18.064 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:17 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:17.716+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:43.284051+0000 front 2024-09-17T08:40:43.284038+0000 (oldest deadline 2024-09-17T08:41:13.884456+0000) 2024-09-17T08:41:18.065 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:17 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:17.840+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:35.995970+0000 front 2024-09-17T08:40:35.996185+0000 (oldest deadline 2024-09-17T08:41:01.896184+0000) 2024-09-17T08:41:18.065 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:17 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:17.840+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:44.697871+0000 front 2024-09-17T08:40:44.697971+0000 (oldest deadline 2024-09-17T08:41:09.397883+0000) 2024-09-17T08:41:18.065 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:17 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:17.840+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-09-17T08:40:54.599304+0000 front 2024-09-17T08:40:54.599361+0000 (oldest deadline 2024-09-17T08:41:16.899409+0000) 2024-09-17T08:41:18.395 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:18 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5[44539]: 2024-09-17T08:41:18.064+0000 7f21d8ede640 -1 osd.5 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:39.673962+0000 front 2024-09-17T08:40:39.674054+0000 (oldest deadline 2024-09-17T08:41:01.373856+0000) 2024-09-17T08:41:18.395 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:18 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5[44539]: 2024-09-17T08:41:18.064+0000 7f21d8ede640 -1 osd.5 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:47.275114+0000 front 2024-09-17T08:40:47.275068+0000 (oldest deadline 2024-09-17T08:41:10.174959+0000) 2024-09-17T08:41:18.395 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:18 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5[44539]: 2024-09-17T08:41:18.064+0000 7f21d8ede640 -1 osd.5 74 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-09-17T08:40:52.476009+0000 front 2024-09-17T08:40:52.475922+0000 (oldest deadline 2024-09-17T08:41:17.175838+0000) 2024-09-17T08:41:19.063 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:18 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:18.719+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:37.983615+0000 front 2024-09-17T08:40:37.983624+0000 (oldest deadline 2024-09-17T08:41:03.283463+0000) 2024-09-17T08:41:19.063 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:18 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:18.719+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:43.284051+0000 front 2024-09-17T08:40:43.284038+0000 (oldest deadline 2024-09-17T08:41:13.884456+0000) 2024-09-17T08:41:19.064 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:18 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:18.873+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:35.995970+0000 front 2024-09-17T08:40:35.996185+0000 (oldest deadline 2024-09-17T08:41:01.896184+0000) 2024-09-17T08:41:19.064 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:18 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:18.873+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:44.697871+0000 front 2024-09-17T08:40:44.697971+0000 (oldest deadline 2024-09-17T08:41:09.397883+0000) 2024-09-17T08:41:19.064 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:18 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:18.873+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-09-17T08:40:54.599304+0000 front 2024-09-17T08:40:54.599361+0000 (oldest deadline 2024-09-17T08:41:16.899409+0000) 2024-09-17T08:41:19.395 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:19 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5[44539]: 2024-09-17T08:41:19.063+0000 7f21d8ede640 -1 osd.5 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:39.673962+0000 front 2024-09-17T08:40:39.674054+0000 (oldest deadline 2024-09-17T08:41:01.373856+0000) 2024-09-17T08:41:19.395 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:19 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5[44539]: 2024-09-17T08:41:19.063+0000 7f21d8ede640 -1 osd.5 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:47.275114+0000 front 2024-09-17T08:40:47.275068+0000 (oldest deadline 2024-09-17T08:41:10.174959+0000) 2024-09-17T08:41:19.395 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:19 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5[44539]: 2024-09-17T08:41:19.063+0000 7f21d8ede640 -1 osd.5 74 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-09-17T08:40:52.476009+0000 front 2024-09-17T08:40:52.475922+0000 (oldest deadline 2024-09-17T08:41:17.175838+0000) 2024-09-17T08:41:20.123 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:19 smithi170 podman[73541]: 2024-09-17 08:41:19.840210202 +0000 UTC m=+5.461724531 container died 4061e465e88b02f7970fe5f0cadc98daaeeeee720a3fedfba0829424ece530e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T08:41:20.123 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:19 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:19.727+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:37.983615+0000 front 2024-09-17T08:40:37.983624+0000 (oldest deadline 2024-09-17T08:41:03.283463+0000) 2024-09-17T08:41:20.124 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:19 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:19.727+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:43.284051+0000 front 2024-09-17T08:40:43.284038+0000 (oldest deadline 2024-09-17T08:41:13.884456+0000) 2024-09-17T08:41:20.124 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:19 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:19.893+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:35.995970+0000 front 2024-09-17T08:40:35.996185+0000 (oldest deadline 2024-09-17T08:41:01.896184+0000) 2024-09-17T08:41:20.124 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:19 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:19.893+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:44.697871+0000 front 2024-09-17T08:40:44.697971+0000 (oldest deadline 2024-09-17T08:41:09.397883+0000) 2024-09-17T08:41:20.124 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:19 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:19.893+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-09-17T08:40:54.599304+0000 front 2024-09-17T08:40:54.599361+0000 (oldest deadline 2024-09-17T08:41:16.899409+0000) 2024-09-17T08:41:20.395 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:20 smithi170 podman[73541]: 2024-09-17 08:41:20.222221959 +0000 UTC m=+5.843736290 container cleanup 4061e465e88b02f7970fe5f0cadc98daaeeeee720a3fedfba0829424ece530e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5, GIT_CLEAN=True, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-09-17T08:41:20.395 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:20 smithi170 bash[73541]: ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5 2024-09-17T08:41:20.751 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:20 smithi170 podman[73562]: 2024-09-17 08:41:20.498482085 +0000 UTC m=+0.651389577 container remove 4061e465e88b02f7970fe5f0cadc98daaeeeee720a3fedfba0829424ece530e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, io.buildah.version=1.37.2) 2024-09-17T08:41:21.013 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:20 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:20.925+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:35.995970+0000 front 2024-09-17T08:40:35.996185+0000 (oldest deadline 2024-09-17T08:41:01.896184+0000) 2024-09-17T08:41:21.013 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:20 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:20.925+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:44.697871+0000 front 2024-09-17T08:40:44.697971+0000 (oldest deadline 2024-09-17T08:41:09.397883+0000) 2024-09-17T08:41:21.014 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:20 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:20.925+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-09-17T08:40:54.599304+0000 front 2024-09-17T08:40:54.599361+0000 (oldest deadline 2024-09-17T08:41:16.899409+0000) 2024-09-17T08:41:21.014 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:20 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:20.750+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:37.983615+0000 front 2024-09-17T08:40:37.983624+0000 (oldest deadline 2024-09-17T08:41:03.283463+0000) 2024-09-17T08:41:21.014 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:20 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:20.750+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:43.284051+0000 front 2024-09-17T08:40:43.284038+0000 (oldest deadline 2024-09-17T08:41:13.884456+0000) 2024-09-17T08:41:21.014 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:20 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:20.750+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-09-17T08:40:53.885039+0000 front 2024-09-17T08:40:53.885054+0000 (oldest deadline 2024-09-17T08:41:19.785036+0000) 2024-09-17T08:41:21.014 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:20 smithi170 podman[73657]: 2024-09-17 08:41:20.709287984 +0000 UTC m=+0.015944134 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T08:41:21.014 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:20 smithi170 podman[73657]: 2024-09-17 08:41:20.816857025 +0000 UTC m=+0.123513203 container create f06a516a9dafd4284951267cfa4d3fc93f56e6b7729d469d4cb7ce8d302c591c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T08:41:21.014 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:21 smithi170 podman[73657]: 2024-09-17 08:41:21.013836919 +0000 UTC m=+0.320493051 container init f06a516a9dafd4284951267cfa4d3fc93f56e6b7729d469d4cb7ce8d302c591c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-17T08:41:21.396 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:21 smithi170 podman[73657]: 2024-09-17 08:41:21.019817055 +0000 UTC m=+0.326473188 container start f06a516a9dafd4284951267cfa4d3fc93f56e6b7729d469d4cb7ce8d302c591c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-19e2a2b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-17T08:41:21.396 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:21 smithi170 podman[73657]: 2024-09-17 08:41:21.105999236 +0000 UTC m=+0.412655386 container attach f06a516a9dafd4284951267cfa4d3fc93f56e6b7729d469d4cb7ce8d302c591c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-17T08:41:21.396 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 17 08:41:21 smithi170 podman[73657]: 2024-09-17 08:41:21.205140548 +0000 UTC m=+0.511796693 container died f06a516a9dafd4284951267cfa4d3fc93f56e6b7729d469d4cb7ce8d302c591c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-5-deactivate, ceph=True, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-17T08:41:21.743 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:21 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:21.725+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:37.983615+0000 front 2024-09-17T08:40:37.983624+0000 (oldest deadline 2024-09-17T08:41:03.283463+0000) 2024-09-17T08:41:21.743 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:21 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:21.725+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:43.284051+0000 front 2024-09-17T08:40:43.284038+0000 (oldest deadline 2024-09-17T08:41:13.884456+0000) 2024-09-17T08:41:21.743 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:21 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:21.725+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-09-17T08:40:53.885039+0000 front 2024-09-17T08:40:53.885054+0000 (oldest deadline 2024-09-17T08:41:19.785036+0000) 2024-09-17T08:41:21.758 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7862d192-74ce-11ef-bceb-c7b262605968@osd.5.service' 2024-09-17T08:41:21.797 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:41:21.797 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-17T08:41:21.797 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-17T08:41:21.797 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-7862d192-74ce-11ef-bceb-c7b262605968@osd.6 2024-09-17T08:41:22.131 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:21 smithi170 systemd[1]: Stopping Ceph osd.6 for 7862d192-74ce-11ef-bceb-c7b262605968... 2024-09-17T08:41:22.132 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:21 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:21.951+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:35.995970+0000 front 2024-09-17T08:40:35.996185+0000 (oldest deadline 2024-09-17T08:41:01.896184+0000) 2024-09-17T08:41:22.132 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:21 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:21.951+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:44.697871+0000 front 2024-09-17T08:40:44.697971+0000 (oldest deadline 2024-09-17T08:41:09.397883+0000) 2024-09-17T08:41:22.132 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:21 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:21.951+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-09-17T08:40:54.599304+0000 front 2024-09-17T08:40:54.599361+0000 (oldest deadline 2024-09-17T08:41:16.899409+0000) 2024-09-17T08:41:22.132 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:22.131+0000 7fcaab3a2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:41:22.132 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:22.131+0000 7fcaab3a2640 -1 osd.6 74 *** Got signal Terminated *** 2024-09-17T08:41:22.132 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:22.131+0000 7fcaab3a2640 -1 osd.6 74 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T08:41:23.145 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:22.909+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:35.995970+0000 front 2024-09-17T08:40:35.996185+0000 (oldest deadline 2024-09-17T08:41:01.896184+0000) 2024-09-17T08:41:23.145 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:22.909+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:44.697871+0000 front 2024-09-17T08:40:44.697971+0000 (oldest deadline 2024-09-17T08:41:09.397883+0000) 2024-09-17T08:41:23.145 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:22.909+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-09-17T08:40:54.599304+0000 front 2024-09-17T08:40:54.599361+0000 (oldest deadline 2024-09-17T08:41:16.899409+0000) 2024-09-17T08:41:23.146 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:22.748+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:37.983615+0000 front 2024-09-17T08:40:37.983624+0000 (oldest deadline 2024-09-17T08:41:03.283463+0000) 2024-09-17T08:41:23.146 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:22.748+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:43.284051+0000 front 2024-09-17T08:40:43.284038+0000 (oldest deadline 2024-09-17T08:41:13.884456+0000) 2024-09-17T08:41:23.146 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:22 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:22.748+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-09-17T08:40:53.885039+0000 front 2024-09-17T08:40:53.885054+0000 (oldest deadline 2024-09-17T08:41:19.785036+0000) 2024-09-17T08:41:24.145 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:23 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:23.882+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:35.995970+0000 front 2024-09-17T08:40:35.996185+0000 (oldest deadline 2024-09-17T08:41:01.896184+0000) 2024-09-17T08:41:24.145 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:23 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:23.882+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:44.697871+0000 front 2024-09-17T08:40:44.697971+0000 (oldest deadline 2024-09-17T08:41:09.397883+0000) 2024-09-17T08:41:24.145 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:23 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:23.882+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-09-17T08:40:54.599304+0000 front 2024-09-17T08:40:54.599361+0000 (oldest deadline 2024-09-17T08:41:16.899409+0000) 2024-09-17T08:41:24.146 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:23 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:23.765+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:37.983615+0000 front 2024-09-17T08:40:37.983624+0000 (oldest deadline 2024-09-17T08:41:03.283463+0000) 2024-09-17T08:41:24.146 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:23 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:23.765+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:43.284051+0000 front 2024-09-17T08:40:43.284038+0000 (oldest deadline 2024-09-17T08:41:13.884456+0000) 2024-09-17T08:41:24.146 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:23 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:23.765+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-09-17T08:40:53.885039+0000 front 2024-09-17T08:40:53.885054+0000 (oldest deadline 2024-09-17T08:41:19.785036+0000) 2024-09-17T08:41:25.145 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:24 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:24.892+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:35.995970+0000 front 2024-09-17T08:40:35.996185+0000 (oldest deadline 2024-09-17T08:41:01.896184+0000) 2024-09-17T08:41:25.145 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:24 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:24.892+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:44.697871+0000 front 2024-09-17T08:40:44.697971+0000 (oldest deadline 2024-09-17T08:41:09.397883+0000) 2024-09-17T08:41:25.145 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:24 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:24.892+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-09-17T08:40:54.599304+0000 front 2024-09-17T08:40:54.599361+0000 (oldest deadline 2024-09-17T08:41:16.899409+0000) 2024-09-17T08:41:25.146 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:24 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:24.749+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:37.983615+0000 front 2024-09-17T08:40:37.983624+0000 (oldest deadline 2024-09-17T08:41:03.283463+0000) 2024-09-17T08:41:25.146 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:24 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:24.749+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:43.284051+0000 front 2024-09-17T08:40:43.284038+0000 (oldest deadline 2024-09-17T08:41:13.884456+0000) 2024-09-17T08:41:25.146 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:24 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:24.749+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-09-17T08:40:53.885039+0000 front 2024-09-17T08:40:53.885054+0000 (oldest deadline 2024-09-17T08:41:19.785036+0000) 2024-09-17T08:41:25.146 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:24 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:24.749+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6830 osd.3 since back 2024-09-17T08:41:00.885986+0000 front 2024-09-17T08:41:00.886120+0000 (oldest deadline 2024-09-17T08:41:24.385919+0000) 2024-09-17T08:41:26.145 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:25 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:25.853+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:35.995970+0000 front 2024-09-17T08:40:35.996185+0000 (oldest deadline 2024-09-17T08:41:01.896184+0000) 2024-09-17T08:41:26.145 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:25 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:25.853+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:44.697871+0000 front 2024-09-17T08:40:44.697971+0000 (oldest deadline 2024-09-17T08:41:09.397883+0000) 2024-09-17T08:41:26.145 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:25 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:25.853+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-09-17T08:40:54.599304+0000 front 2024-09-17T08:40:54.599361+0000 (oldest deadline 2024-09-17T08:41:16.899409+0000) 2024-09-17T08:41:26.146 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:25 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:25.748+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:37.983615+0000 front 2024-09-17T08:40:37.983624+0000 (oldest deadline 2024-09-17T08:41:03.283463+0000) 2024-09-17T08:41:26.146 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:25 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:25.748+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:43.284051+0000 front 2024-09-17T08:40:43.284038+0000 (oldest deadline 2024-09-17T08:41:13.884456+0000) 2024-09-17T08:41:26.146 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:25 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:25.748+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-09-17T08:40:53.885039+0000 front 2024-09-17T08:40:53.885054+0000 (oldest deadline 2024-09-17T08:41:19.785036+0000) 2024-09-17T08:41:26.146 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:25 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:25.748+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6830 osd.3 since back 2024-09-17T08:41:00.885986+0000 front 2024-09-17T08:41:00.886120+0000 (oldest deadline 2024-09-17T08:41:24.385919+0000) 2024-09-17T08:41:27.145 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:26 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:26.844+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:35.995970+0000 front 2024-09-17T08:40:35.996185+0000 (oldest deadline 2024-09-17T08:41:01.896184+0000) 2024-09-17T08:41:27.145 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:26 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:26.844+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:44.697871+0000 front 2024-09-17T08:40:44.697971+0000 (oldest deadline 2024-09-17T08:41:09.397883+0000) 2024-09-17T08:41:27.146 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:26 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6[50417]: 2024-09-17T08:41:26.844+0000 7fcaa79b6640 -1 osd.6 74 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-09-17T08:40:54.599304+0000 front 2024-09-17T08:40:54.599361+0000 (oldest deadline 2024-09-17T08:41:16.899409+0000) 2024-09-17T08:41:27.146 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:26 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:26.768+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:37.983615+0000 front 2024-09-17T08:40:37.983624+0000 (oldest deadline 2024-09-17T08:41:03.283463+0000) 2024-09-17T08:41:27.146 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:26 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:26.768+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:43.284051+0000 front 2024-09-17T08:40:43.284038+0000 (oldest deadline 2024-09-17T08:41:13.884456+0000) 2024-09-17T08:41:27.146 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:26 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:26.768+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-09-17T08:40:53.885039+0000 front 2024-09-17T08:40:53.885054+0000 (oldest deadline 2024-09-17T08:41:19.785036+0000) 2024-09-17T08:41:27.146 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:26 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:26.768+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6830 osd.3 since back 2024-09-17T08:41:00.885986+0000 front 2024-09-17T08:41:00.886120+0000 (oldest deadline 2024-09-17T08:41:24.385919+0000) 2024-09-17T08:41:27.783 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:27 smithi170 podman[73786]: 2024-09-17 08:41:27.52253262 +0000 UTC m=+5.496499323 container died 92f0d5fa81fff5ce83e7e6bde4c33d8d1688bd6634ac069d76a7c1b782c0d0e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-17T08:41:28.145 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:27 smithi170 podman[73786]: 2024-09-17 08:41:27.895756106 +0000 UTC m=+5.869722807 container cleanup 92f0d5fa81fff5ce83e7e6bde4c33d8d1688bd6634ac069d76a7c1b782c0d0e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-17T08:41:28.145 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:27 smithi170 bash[73786]: ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6 2024-09-17T08:41:28.146 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:27 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:27.784+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:37.983615+0000 front 2024-09-17T08:40:37.983624+0000 (oldest deadline 2024-09-17T08:41:03.283463+0000) 2024-09-17T08:41:28.146 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:27 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:27.784+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:43.284051+0000 front 2024-09-17T08:40:43.284038+0000 (oldest deadline 2024-09-17T08:41:13.884456+0000) 2024-09-17T08:41:28.146 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:27 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:27.784+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-09-17T08:40:53.885039+0000 front 2024-09-17T08:40:53.885054+0000 (oldest deadline 2024-09-17T08:41:19.785036+0000) 2024-09-17T08:41:28.146 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:27 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:27.784+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6830 osd.3 since back 2024-09-17T08:41:00.885986+0000 front 2024-09-17T08:41:00.886120+0000 (oldest deadline 2024-09-17T08:41:24.385919+0000) 2024-09-17T08:41:28.499 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:28 smithi170 podman[73809]: 2024-09-17 08:41:28.171920676 +0000 UTC m=+0.643136698 container remove 92f0d5fa81fff5ce83e7e6bde4c33d8d1688bd6634ac069d76a7c1b782c0d0e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True) 2024-09-17T08:41:28.780 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:28 smithi170 podman[73904]: 2024-09-17 08:41:28.402014951 +0000 UTC m=+0.015420836 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T08:41:28.780 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:28 smithi170 podman[73904]: 2024-09-17 08:41:28.511189654 +0000 UTC m=+0.124595521 container create 437edf94b08776772a90736e1c6858b5ff32e3a9edb4d9f74e92c69fa6b8ca7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T08:41:28.780 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:28 smithi170 podman[73904]: 2024-09-17 08:41:28.697162622 +0000 UTC m=+0.310568497 container init 437edf94b08776772a90736e1c6858b5ff32e3a9edb4d9f74e92c69fa6b8ca7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-17T08:41:28.780 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:28 smithi170 podman[73904]: 2024-09-17 08:41:28.702447101 +0000 UTC m=+0.315852964 container start 437edf94b08776772a90736e1c6858b5ff32e3a9edb4d9f74e92c69fa6b8ca7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b) 2024-09-17T08:41:28.780 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:28 smithi170 podman[73904]: 2024-09-17 08:41:28.78018727 +0000 UTC m=+0.393593133 container attach 437edf94b08776772a90736e1c6858b5ff32e3a9edb4d9f74e92c69fa6b8ca7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=reef-19e2a2b) 2024-09-17T08:41:29.145 INFO:journalctl@ceph.osd.6.smithi170.stdout:Sep 17 08:41:28 smithi170 podman[73904]: 2024-09-17 08:41:28.888905602 +0000 UTC m=+0.502311476 container died 437edf94b08776772a90736e1c6858b5ff32e3a9edb4d9f74e92c69fa6b8ca7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731) 2024-09-17T08:41:29.146 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:28 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:28.797+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:37.983615+0000 front 2024-09-17T08:40:37.983624+0000 (oldest deadline 2024-09-17T08:41:03.283463+0000) 2024-09-17T08:41:29.146 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:28 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:28.797+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:43.284051+0000 front 2024-09-17T08:40:43.284038+0000 (oldest deadline 2024-09-17T08:41:13.884456+0000) 2024-09-17T08:41:29.146 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:28 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:28.797+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-09-17T08:40:53.885039+0000 front 2024-09-17T08:40:53.885054+0000 (oldest deadline 2024-09-17T08:41:19.785036+0000) 2024-09-17T08:41:29.146 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:28 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:28.797+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6830 osd.3 since back 2024-09-17T08:41:00.885986+0000 front 2024-09-17T08:41:00.886120+0000 (oldest deadline 2024-09-17T08:41:24.385919+0000) 2024-09-17T08:41:29.471 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7862d192-74ce-11ef-bceb-c7b262605968@osd.6.service' 2024-09-17T08:41:29.509 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:41:29.509 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-17T08:41:29.510 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-17T08:41:29.510 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-7862d192-74ce-11ef-bceb-c7b262605968@osd.7 2024-09-17T08:41:29.816 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:29 smithi170 systemd[1]: Stopping Ceph osd.7 for 7862d192-74ce-11ef-bceb-c7b262605968... 2024-09-17T08:41:30.148 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:29 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:29.816+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:37.983615+0000 front 2024-09-17T08:40:37.983624+0000 (oldest deadline 2024-09-17T08:41:03.283463+0000) 2024-09-17T08:41:30.148 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:29 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:29.816+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:43.284051+0000 front 2024-09-17T08:40:43.284038+0000 (oldest deadline 2024-09-17T08:41:13.884456+0000) 2024-09-17T08:41:30.148 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:29 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:29.816+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-09-17T08:40:53.885039+0000 front 2024-09-17T08:40:53.885054+0000 (oldest deadline 2024-09-17T08:41:19.785036+0000) 2024-09-17T08:41:30.148 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:29 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:29.816+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6830 osd.3 since back 2024-09-17T08:41:00.885986+0000 front 2024-09-17T08:41:00.886120+0000 (oldest deadline 2024-09-17T08:41:24.385919+0000) 2024-09-17T08:41:30.148 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:29 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:29.848+0000 7faf74206640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:41:30.149 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:29 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:29.848+0000 7faf74206640 -1 osd.7 74 *** Got signal Terminated *** 2024-09-17T08:41:30.149 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:29 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:29.848+0000 7faf74206640 -1 osd.7 74 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T08:41:31.145 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:30 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:30.822+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:37.983615+0000 front 2024-09-17T08:40:37.983624+0000 (oldest deadline 2024-09-17T08:41:03.283463+0000) 2024-09-17T08:41:31.145 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:30 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:30.822+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:43.284051+0000 front 2024-09-17T08:40:43.284038+0000 (oldest deadline 2024-09-17T08:41:13.884456+0000) 2024-09-17T08:41:31.145 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:30 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:30.822+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-09-17T08:40:53.885039+0000 front 2024-09-17T08:40:53.885054+0000 (oldest deadline 2024-09-17T08:41:19.785036+0000) 2024-09-17T08:41:31.145 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:30 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:30.822+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6830 osd.3 since back 2024-09-17T08:41:00.885986+0000 front 2024-09-17T08:41:00.886120+0000 (oldest deadline 2024-09-17T08:41:24.385919+0000) 2024-09-17T08:41:32.145 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:31 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:31.846+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:37.983615+0000 front 2024-09-17T08:40:37.983624+0000 (oldest deadline 2024-09-17T08:41:03.283463+0000) 2024-09-17T08:41:32.145 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:31 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:31.846+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:43.284051+0000 front 2024-09-17T08:40:43.284038+0000 (oldest deadline 2024-09-17T08:41:13.884456+0000) 2024-09-17T08:41:32.145 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:31 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:31.846+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-09-17T08:40:53.885039+0000 front 2024-09-17T08:40:53.885054+0000 (oldest deadline 2024-09-17T08:41:19.785036+0000) 2024-09-17T08:41:32.146 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:31 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:31.846+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6830 osd.3 since back 2024-09-17T08:41:00.885986+0000 front 2024-09-17T08:41:00.886120+0000 (oldest deadline 2024-09-17T08:41:24.385919+0000) 2024-09-17T08:41:33.145 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:32 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:32.827+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:37.983615+0000 front 2024-09-17T08:40:37.983624+0000 (oldest deadline 2024-09-17T08:41:03.283463+0000) 2024-09-17T08:41:33.145 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:32 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:32.827+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:43.284051+0000 front 2024-09-17T08:40:43.284038+0000 (oldest deadline 2024-09-17T08:41:13.884456+0000) 2024-09-17T08:41:33.145 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:32 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:32.827+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-09-17T08:40:53.885039+0000 front 2024-09-17T08:40:53.885054+0000 (oldest deadline 2024-09-17T08:41:19.785036+0000) 2024-09-17T08:41:33.145 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:32 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:32.827+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6830 osd.3 since back 2024-09-17T08:41:00.885986+0000 front 2024-09-17T08:41:00.886120+0000 (oldest deadline 2024-09-17T08:41:24.385919+0000) 2024-09-17T08:41:34.145 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:33 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:33.791+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:37.983615+0000 front 2024-09-17T08:40:37.983624+0000 (oldest deadline 2024-09-17T08:41:03.283463+0000) 2024-09-17T08:41:34.145 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:33 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:33.791+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:43.284051+0000 front 2024-09-17T08:40:43.284038+0000 (oldest deadline 2024-09-17T08:41:13.884456+0000) 2024-09-17T08:41:34.145 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:33 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:33.791+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-09-17T08:40:53.885039+0000 front 2024-09-17T08:40:53.885054+0000 (oldest deadline 2024-09-17T08:41:19.785036+0000) 2024-09-17T08:41:34.146 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:33 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:33.791+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6830 osd.3 since back 2024-09-17T08:41:00.885986+0000 front 2024-09-17T08:41:00.886120+0000 (oldest deadline 2024-09-17T08:41:24.385919+0000) 2024-09-17T08:41:35.145 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:34 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:34.817+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-09-17T08:40:37.983615+0000 front 2024-09-17T08:40:37.983624+0000 (oldest deadline 2024-09-17T08:41:03.283463+0000) 2024-09-17T08:41:35.145 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:34 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:34.817+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-09-17T08:40:43.284051+0000 front 2024-09-17T08:40:43.284038+0000 (oldest deadline 2024-09-17T08:41:13.884456+0000) 2024-09-17T08:41:35.145 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:34 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:34.817+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-09-17T08:40:53.885039+0000 front 2024-09-17T08:40:53.885054+0000 (oldest deadline 2024-09-17T08:41:19.785036+0000) 2024-09-17T08:41:35.145 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:34 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7[56878]: 2024-09-17T08:41:34.817+0000 7faf7081a640 -1 osd.7 74 heartbeat_check: no reply from 172.21.15.118:6830 osd.3 since back 2024-09-17T08:41:00.885986+0000 front 2024-09-17T08:41:00.886120+0000 (oldest deadline 2024-09-17T08:41:24.385919+0000) 2024-09-17T08:41:35.456 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:35 smithi170 podman[74029]: 2024-09-17 08:41:35.160341491 +0000 UTC m=+5.428912920 container died 922efaf249d37ce5e168df307633a4fed6c68e05de539dfe1d960652b2adb833 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=reef-19e2a2b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T08:41:35.811 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:35 smithi170 podman[74029]: 2024-09-17 08:41:35.58681276 +0000 UTC m=+5.855384171 container cleanup 922efaf249d37ce5e168df307633a4fed6c68e05de539dfe1d960652b2adb833 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, RELEASE=reef-19e2a2b, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-17T08:41:35.811 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:35 smithi170 bash[74029]: ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7 2024-09-17T08:41:36.145 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:35 smithi170 podman[74061]: 2024-09-17 08:41:35.811868623 +0000 UTC m=+0.642295347 container remove 922efaf249d37ce5e168df307633a4fed6c68e05de539dfe1d960652b2adb833 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7, RELEASE=reef-19e2a2b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T08:41:36.548 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:36 smithi170 podman[74157]: 2024-09-17 08:41:36.108067905 +0000 UTC m=+0.015832591 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T08:41:36.549 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:36 smithi170 podman[74157]: 2024-09-17 08:41:36.212759824 +0000 UTC m=+0.120524507 container create 02f287c1968e1a998324828d6253734c495f01237a55fefb14d58369e791a4f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=reef-19e2a2b, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-17T08:41:36.549 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:36 smithi170 podman[74157]: 2024-09-17 08:41:36.424072882 +0000 UTC m=+0.331837568 container init 02f287c1968e1a998324828d6253734c495f01237a55fefb14d58369e791a4f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-17T08:41:36.549 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:36 smithi170 podman[74157]: 2024-09-17 08:41:36.429082825 +0000 UTC m=+0.336847514 container start 02f287c1968e1a998324828d6253734c495f01237a55fefb14d58369e791a4f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-17T08:41:36.549 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:36 smithi170 podman[74157]: 2024-09-17 08:41:36.548979396 +0000 UTC m=+0.456744081 container attach 02f287c1968e1a998324828d6253734c495f01237a55fefb14d58369e791a4f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=reef-19e2a2b, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True) 2024-09-17T08:41:36.828 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 17 08:41:36 smithi170 podman[74157]: 2024-09-17 08:41:36.612532691 +0000 UTC m=+0.520297375 container died 02f287c1968e1a998324828d6253734c495f01237a55fefb14d58369e791a4f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-osd-7-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-17T08:41:37.137 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7862d192-74ce-11ef-bceb-c7b262605968@osd.7.service' 2024-09-17T08:41:37.176 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:41:37.177 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-17T08:41:37.177 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-09-17T08:41:37.177 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ceph-7862d192-74ce-11ef-bceb-c7b262605968@rgw.foo.a 2024-09-17T08:41:37.483 INFO:journalctl@ceph.rgw.foo.a.smithi118.stdout:Sep 17 08:41:37 smithi118 systemd[1]: Stopping Ceph rgw.foo.a for 7862d192-74ce-11ef-bceb-c7b262605968... 2024-09-17T08:41:37.777 INFO:journalctl@ceph.rgw.foo.a.smithi118.stdout:Sep 17 08:41:37 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-rgw-foo-a[69347]: 2024-09-17T08:41:37.482+0000 7fdd37df9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:41:37.777 INFO:journalctl@ceph.rgw.foo.a.smithi118.stdout:Sep 17 08:41:37 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-rgw-foo-a[69347]: 2024-09-17T08:41:37.482+0000 7fdd3b664880 -1 shutting down 2024-09-17T08:41:42.027 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:41:41 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a[79253]: ts=2024-09-17T08:41:41.625Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi118.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.118:8443: connect: connection refused" 2024-09-17T08:41:42.027 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:41:41 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a[79253]: ts=2024-09-17T08:41:41.625Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi170.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.170:8443: connect: connection refused" 2024-09-17T08:41:47.777 INFO:journalctl@ceph.rgw.foo.a.smithi118.stdout:Sep 17 08:41:47 smithi118 bash[88682]: time="2024-09-17T08:41:47Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-7862d192-74ce-11ef-bceb-c7b262605968-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-09-17T08:41:47.778 INFO:journalctl@ceph.rgw.foo.a.smithi118.stdout:Sep 17 08:41:47 smithi118 podman[88682]: 2024-09-17 08:41:47.503206653 +0000 UTC m=+10.131234949 container died 48b9ec86736d0c21eccb98dfcaa4e1edcf6564f71ee13830c666bf3a410fc939 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-rgw-foo-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-17T08:41:48.193 INFO:journalctl@ceph.rgw.foo.a.smithi118.stdout:Sep 17 08:41:47 smithi118 podman[88682]: 2024-09-17 08:41:47.954385084 +0000 UTC m=+10.582413375 container cleanup 48b9ec86736d0c21eccb98dfcaa4e1edcf6564f71ee13830c666bf3a410fc939 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-rgw-foo-a, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, RELEASE=reef-19e2a2b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-17T08:41:48.193 INFO:journalctl@ceph.rgw.foo.a.smithi118.stdout:Sep 17 08:41:47 smithi118 bash[88682]: ceph-7862d192-74ce-11ef-bceb-c7b262605968-rgw-foo-a 2024-09-17T08:41:48.388 DEBUG:teuthology.orchestra.run.smithi118:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7862d192-74ce-11ef-bceb-c7b262605968@rgw.foo.a.service' 2024-09-17T08:41:48.424 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:41:48.424 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-09-17T08:41:48.424 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-17T08:41:48.424 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-7862d192-74ce-11ef-bceb-c7b262605968@prometheus.a 2024-09-17T08:41:48.707 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:41:48 smithi170 systemd[1]: Stopping Ceph prometheus.a for 7862d192-74ce-11ef-bceb-c7b262605968... 2024-09-17T08:41:48.963 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:41:48 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:41:48.706Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-17T08:41:48.964 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:41:48 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:41:48.707Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-17T08:41:48.964 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:41:48 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:41:48.707Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-17T08:41:48.964 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:41:48 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:41:48.707Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-17T08:41:48.964 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:41:48 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:41:48.707Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-17T08:41:48.964 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:41:48 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:41:48.707Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-17T08:41:48.964 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:41:48 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:41:48.707Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-17T08:41:48.964 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:41:48 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:41:48.707Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-17T08:41:48.964 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:41:48 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:41:48.707Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-17T08:41:48.964 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:41:48 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:41:48.707Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-17T08:41:48.964 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:41:48 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:41:48.707Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-17T08:41:48.964 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:41:48 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a[67418]: ts=2024-09-17T08:41:48.707Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-17T08:41:48.965 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:41:48 smithi170 podman[74284]: 2024-09-17 08:41:48.717401431 +0000 UTC m=+0.103563492 container died 993a3bf4ddb66824af98c86898194a7f16b2386a8d7937a9a363f1aa6fefb0bb (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T08:41:49.339 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:41:49 smithi170 podman[74284]: 2024-09-17 08:41:49.071439028 +0000 UTC m=+0.457601082 container cleanup 993a3bf4ddb66824af98c86898194a7f16b2386a8d7937a9a363f1aa6fefb0bb (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T08:41:49.340 INFO:journalctl@ceph.prometheus.a.smithi170.stdout:Sep 17 08:41:49 smithi170 bash[74284]: ceph-7862d192-74ce-11ef-bceb-c7b262605968-prometheus-a 2024-09-17T08:41:49.482 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7862d192-74ce-11ef-bceb-c7b262605968@prometheus.a.service' 2024-09-17T08:41:49.517 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:41:49.517 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-17T08:41:49.517 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm rm-cluster --fsid 7862d192-74ce-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-17T08:41:49.667 INFO:teuthology.orchestra.run.smithi118.stdout:Deleting cluster with fsid: 7862d192-74ce-11ef-bceb-c7b262605968 2024-09-17T08:41:51.277 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:41:51 smithi118 systemd[1]: Stopping Ceph alertmanager.a for 7862d192-74ce-11ef-bceb-c7b262605968... 2024-09-17T08:41:51.652 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:41:51 smithi118 ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a[79253]: ts=2024-09-17T08:41:51.328Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-17T08:41:51.652 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:41:51 smithi118 podman[89155]: 2024-09-17 08:41:51.39965274 +0000 UTC m=+0.177211205 container died 8b804409f7af67cbd06b25eb49686ff2a98463d016f2d60cfe3c31337326c925 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T08:41:52.027 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:41:51 smithi118 podman[89155]: 2024-09-17 08:41:51.742646785 +0000 UTC m=+0.520205249 container cleanup 8b804409f7af67cbd06b25eb49686ff2a98463d016f2d60cfe3c31337326c925 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T08:41:52.027 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:41:51 smithi118 bash[89155]: ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a 2024-09-17T08:41:52.299 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:41:52 smithi118 podman[89172]: 2024-09-17 08:41:52.043897974 +0000 UTC m=+0.646284330 container remove 8b804409f7af67cbd06b25eb49686ff2a98463d016f2d60cfe3c31337326c925 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T08:41:52.299 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:41:52 smithi118 podman[89172]: 2024-09-17 08:41:52.127956731 +0000 UTC m=+0.730343089 volume remove 19a09ff9a6638b823c47a9f3ec1d090c6f119c684f680acc9d4c4fa6ec6e9d81 2024-09-17T08:41:52.742 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:41:52 smithi118 systemd[1]: Stopping Ceph node-exporter.a for 7862d192-74ce-11ef-bceb-c7b262605968... 2024-09-17T08:41:52.743 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:41:52 smithi118 systemd[1]: ceph-7862d192-74ce-11ef-bceb-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-17T08:41:52.743 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:41:52 smithi118 systemd[1]: Stopped Ceph alertmanager.a for 7862d192-74ce-11ef-bceb-c7b262605968. 2024-09-17T08:41:52.743 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Sep 17 08:41:52 smithi118 systemd[1]: ceph-7862d192-74ce-11ef-bceb-c7b262605968@alertmanager.a.service: Consumed 1.267s CPU time. 2024-09-17T08:41:53.027 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:41:52 smithi118 podman[89287]: 2024-09-17 08:41:52.743206567 +0000 UTC m=+0.119515281 container died d20995b52b5afacca5db23658cba21ddb8e394fc3ed6ec3c1f1619e3465d3b65 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-17T08:41:53.463 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:41:53 smithi118 podman[89287]: 2024-09-17 08:41:53.19691618 +0000 UTC m=+0.573224892 container cleanup d20995b52b5afacca5db23658cba21ddb8e394fc3ed6ec3c1f1619e3465d3b65 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-17T08:41:53.464 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:41:53 smithi118 bash[89287]: ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a 2024-09-17T08:41:53.750 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:41:53 smithi118 podman[89307]: 2024-09-17 08:41:53.46432535 +0000 UTC m=+0.722443256 container remove d20995b52b5afacca5db23658cba21ddb8e394fc3ed6ec3c1f1619e3465d3b65 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-17T08:41:53.751 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:41:53 smithi118 systemd[1]: ceph-7862d192-74ce-11ef-bceb-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-17T08:41:53.751 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:41:53 smithi118 systemd[1]: ceph-7862d192-74ce-11ef-bceb-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-17T08:41:53.751 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:41:53 smithi118 systemd[1]: Stopped Ceph node-exporter.a for 7862d192-74ce-11ef-bceb-c7b262605968. 2024-09-17T08:41:53.751 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Sep 17 08:41:53 smithi118 systemd[1]: ceph-7862d192-74ce-11ef-bceb-c7b262605968@node-exporter.a.service: Consumed 2.558s CPU time. 2024-09-17T08:41:54.207 DEBUG:teuthology.orchestra.run.smithi170:> sudo cephadm rm-cluster --fsid 7862d192-74ce-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-17T08:41:54.361 INFO:teuthology.orchestra.run.smithi170.stdout:Deleting cluster with fsid: 7862d192-74ce-11ef-bceb-c7b262605968 2024-09-17T08:41:55.645 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:41:55 smithi170 systemd[1]: Stopping Ceph iscsi.iscsi.a for 7862d192-74ce-11ef-bceb-c7b262605968... 2024-09-17T08:41:56.145 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:41:55 smithi170 ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a[62012]: debug Shutdown received 2024-09-17T08:42:06.051 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:42:05 smithi170 bash[74671]: time="2024-09-17T08:42:05Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-09-17T08:42:06.052 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:42:05 smithi170 podman[74671]: 2024-09-17 08:42:05.771091557 +0000 UTC m=+10.139611110 container died 67e86fcc3bc37e17cf7fdbe7100de8d1ae0f996b42bfc67e7eba6405d98d3c4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-17T08:42:06.393 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:42:06 smithi170 podman[74671]: 2024-09-17 08:42:06.134080845 +0000 UTC m=+10.502600396 container cleanup 67e86fcc3bc37e17cf7fdbe7100de8d1ae0f996b42bfc67e7eba6405d98d3c4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD) 2024-09-17T08:42:06.393 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:42:06 smithi170 bash[74671]: ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a 2024-09-17T08:42:06.645 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:42:06 smithi170 podman[74695]: 2024-09-17 08:42:06.393545284 +0000 UTC m=+0.618383010 container remove 67e86fcc3bc37e17cf7fdbe7100de8d1ae0f996b42bfc67e7eba6405d98d3c4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a, RELEASE=reef-19e2a2b, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T08:42:06.645 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:42:06 smithi170 systemd[1]: ceph-7862d192-74ce-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-09-17T08:42:06.645 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:42:06 smithi170 podman[74743]: 2024-09-17 08:42:06.642599187 +0000 UTC m=+0.113349526 container died 381796f80abeac7102a9ba74dc06e379958c590ba8b1191e302c14cf47cfaaf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a-tcmu, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-17T08:42:07.145 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:42:06 smithi170 podman[74743]: 2024-09-17 08:42:06.954618224 +0000 UTC m=+0.425368562 container cleanup 381796f80abeac7102a9ba74dc06e379958c590ba8b1191e302c14cf47cfaaf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a-tcmu, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731) 2024-09-17T08:42:07.145 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:42:06 smithi170 bash[74743]: ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a-tcmu 2024-09-17T08:42:07.146 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:42:06 smithi170 systemd[1]: ceph-7862d192-74ce-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-09-17T08:42:07.146 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:42:06 smithi170 systemd[1]: ceph-7862d192-74ce-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Unit process 61874 (conmon) remains running after unit stopped. 2024-09-17T08:42:07.146 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:42:06 smithi170 systemd[1]: ceph-7862d192-74ce-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Unit process 74759 (podman) remains running after unit stopped. 2024-09-17T08:42:07.146 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:42:06 smithi170 systemd[1]: Stopped Ceph iscsi.iscsi.a for 7862d192-74ce-11ef-bceb-c7b262605968. 2024-09-17T08:42:07.146 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:42:06 smithi170 systemd[1]: ceph-7862d192-74ce-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Consumed 2.683s CPU time. 2024-09-17T08:42:07.428 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:42:07 smithi170 systemd[1]: Stopping Ceph grafana.a for 7862d192-74ce-11ef-bceb-c7b262605968... 2024-09-17T08:42:07.428 INFO:journalctl@ceph.iscsi.iscsi.a.smithi170.stdout:Sep 17 08:42:07 smithi170 podman[74759]: 2024-09-17 08:42:07.182861914 +0000 UTC m=+0.543984905 container remove 381796f80abeac7102a9ba74dc06e379958c590ba8b1191e302c14cf47cfaaf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-iscsi-iscsi-a-tcmu, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-17T08:42:07.895 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:42:07 smithi170 podman[74886]: 2024-09-17 08:42:07.647122089 +0000 UTC m=+0.143442545 container died 1294232fb6913e5048cc7b0afc57c05785244270c58d6c977626836cf86648d4 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a, architecture=x86_64, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, vendor=Red Hat, Inc., io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.expose-services=, release=239.1651231664, build-date=2022-04-29T11:48:11.055897, io.buildah.version=1.32.0, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, maintainer=Paul Cuzner , name=ubi8, com.redhat.component=ubi8-container, summary=Grafana Container configured for Ceph mgr/dashboard integration, version=8.5, io.k8s.display-name=Red Hat Universal Base Image 8, description=Ceph Grafana Container, io.openshift.tags=base rhel8, distribution-scope=public, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-type=git) 2024-09-17T08:42:08.260 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:42:08 smithi170 podman[74886]: 2024-09-17 08:42:08.000568101 +0000 UTC m=+0.496888558 container cleanup 1294232fb6913e5048cc7b0afc57c05785244270c58d6c977626836cf86648d4 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a, description=Ceph Grafana Container, build-date=2022-04-29T11:48:11.055897, io.buildah.version=1.32.0, maintainer=Paul Cuzner , architecture=x86_64, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, version=8.5, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.tags=base rhel8, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, distribution-scope=public, release=239.1651231664, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, name=ubi8, summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-type=git, vendor=Red Hat, Inc., io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.expose-services=, com.redhat.component=ubi8-container) 2024-09-17T08:42:08.260 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:42:08 smithi170 bash[74886]: ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a 2024-09-17T08:42:08.609 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:42:08 smithi170 podman[74911]: 2024-09-17 08:42:08.260284194 +0000 UTC m=+0.615341743 container remove 1294232fb6913e5048cc7b0afc57c05785244270c58d6c977626836cf86648d4 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-grafana-a, io.buildah.version=1.32.0, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, description=Ceph Grafana Container, com.redhat.component=ubi8-container, vcs-type=git, vendor=Red Hat, Inc., build-date=2022-04-29T11:48:11.055897, distribution-scope=public, architecture=x86_64, io.openshift.tags=base rhel8, maintainer=Paul Cuzner , release=239.1651231664, io.k8s.display-name=Red Hat Universal Base Image 8, summary=Grafana Container configured for Ceph mgr/dashboard integration, name=ubi8, version=8.5, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.openshift.expose-services=, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664) 2024-09-17T08:42:08.609 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:42:08 smithi170 systemd[1]: ceph-7862d192-74ce-11ef-bceb-c7b262605968@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-09-17T08:42:08.609 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:42:08 smithi170 systemd[1]: ceph-7862d192-74ce-11ef-bceb-c7b262605968@grafana.a.service: Failed with result 'exit-code'. 2024-09-17T08:42:08.609 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:42:08 smithi170 systemd[1]: Stopped Ceph grafana.a for 7862d192-74ce-11ef-bceb-c7b262605968. 2024-09-17T08:42:08.609 INFO:journalctl@ceph.grafana.a.smithi170.stdout:Sep 17 08:42:08 smithi170 systemd[1]: ceph-7862d192-74ce-11ef-bceb-c7b262605968@grafana.a.service: Consumed 6.861s CPU time. 2024-09-17T08:42:08.895 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:42:08 smithi170 systemd[1]: Stopping Ceph node-exporter.b for 7862d192-74ce-11ef-bceb-c7b262605968... 2024-09-17T08:42:09.173 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:42:08 smithi170 podman[75025]: 2024-09-17 08:42:08.918041239 +0000 UTC m=+0.171974410 container died c1c963b1d7f20ed903ba446157960befc60f047d3ed90b59bf054a2847cba533 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-17T08:42:09.548 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:42:09 smithi170 podman[75025]: 2024-09-17 08:42:09.272270148 +0000 UTC m=+0.526203345 container cleanup c1c963b1d7f20ed903ba446157960befc60f047d3ed90b59bf054a2847cba533 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-17T08:42:09.548 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:42:09 smithi170 bash[75025]: ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b 2024-09-17T08:42:09.834 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:42:09 smithi170 podman[75041]: 2024-09-17 08:42:09.54864817 +0000 UTC m=+0.630957411 container remove c1c963b1d7f20ed903ba446157960befc60f047d3ed90b59bf054a2847cba533 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-7862d192-74ce-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-17T08:42:09.835 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:42:09 smithi170 systemd[1]: ceph-7862d192-74ce-11ef-bceb-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-17T08:42:09.835 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:42:09 smithi170 systemd[1]: ceph-7862d192-74ce-11ef-bceb-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-17T08:42:09.835 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:42:09 smithi170 systemd[1]: Stopped Ceph node-exporter.b for 7862d192-74ce-11ef-bceb-c7b262605968. 2024-09-17T08:42:09.835 INFO:journalctl@ceph.node-exporter.b.smithi170.stdout:Sep 17 08:42:09 smithi170 systemd[1]: ceph-7862d192-74ce-11ef-bceb-c7b262605968@node-exporter.b.service: Consumed 2.518s CPU time. 2024-09-17T08:42:10.316 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-17T08:42:10.350 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-17T08:42:10.378 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-17T08:42:10.379 DEBUG:teuthology.misc:Transferring archived files from smithi118:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908211/remote/smithi118/crash 2024-09-17T08:42:10.380 DEBUG:teuthology.orchestra.run.smithi118:> sudo tar c -f - -C /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/crash -- . 2024-09-17T08:42:10.432 INFO:teuthology.orchestra.run.smithi118.stderr:tar: /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-17T08:42:10.433 INFO:teuthology.orchestra.run.smithi118.stderr:tar: Error is not recoverable: exiting now 2024-09-17T08:42:10.434 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908211/remote/smithi170/crash 2024-09-17T08:42:10.435 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/crash -- . 2024-09-17T08:42:10.474 INFO:teuthology.orchestra.run.smithi170.stderr:tar: /var/lib/ceph/7862d192-74ce-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-17T08:42:10.474 INFO:teuthology.orchestra.run.smithi170.stderr:tar: Error is not recoverable: exiting now 2024-09-17T08:42:10.475 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-17T08:42:10.476 DEBUG:teuthology.orchestra.run.smithi118:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/7862d192-74ce-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-09-17T08:42:10.513 INFO:teuthology.orchestra.run.smithi118.stderr:grep: /var/log/ceph/7862d192-74ce-11ef-bceb-c7b262605968/ceph.log: No such file or directory 2024-09-17T08:42:10.515 INFO:tasks.cephadm:Compressing logs... 2024-09-17T08:42:10.515 DEBUG:teuthology.orchestra.run.smithi118:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T08:42:10.558 DEBUG:teuthology.orchestra.run.smithi170:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T08:42:10.581 INFO:teuthology.orchestra.run.smithi118.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-17T08:42:10.583 INFO:teuthology.orchestra.run.smithi170.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-17T08:42:13.390 INFO:tasks.cephadm:Archiving logs... 2024-09-17T08:42:13.390 DEBUG:teuthology.misc:Transferring archived files from smithi118:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908211/remote/smithi118/log 2024-09-17T08:42:13.391 DEBUG:teuthology.orchestra.run.smithi118:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T08:42:13.786 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908211/remote/smithi170/log 2024-09-17T08:42:13.787 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T08:42:14.052 INFO:tasks.cephadm:Removing cluster... 2024-09-17T08:42:14.052 DEBUG:teuthology.orchestra.run.smithi118:> sudo cephadm rm-cluster --fsid 7862d192-74ce-11ef-bceb-c7b262605968 --force 2024-09-17T08:42:14.193 INFO:teuthology.orchestra.run.smithi118.stdout:Deleting cluster with fsid: 7862d192-74ce-11ef-bceb-c7b262605968 2024-09-17T08:42:14.460 DEBUG:teuthology.orchestra.run.smithi170:> sudo cephadm rm-cluster --fsid 7862d192-74ce-11ef-bceb-c7b262605968 --force 2024-09-17T08:42:14.611 INFO:teuthology.orchestra.run.smithi170.stdout:Deleting cluster with fsid: 7862d192-74ce-11ef-bceb-c7b262605968 2024-09-17T08:42:14.877 INFO:tasks.cephadm:Teardown complete 2024-09-17T08:42:14.877 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-17T08:42:14.886 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-17T08:42:14.886 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T08:42:14.889 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T08:42:14.967 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-17T08:42:14.976 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-17T08:42:14.976 DEBUG:teuthology.orchestra.run.smithi118:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T08:42:14.979 DEBUG:teuthology.orchestra.run.smithi170:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T08:42:14.991 INFO:teuthology.orchestra.run.smithi118.stderr:bash: line 1: ntpq: command not found 2024-09-17T08:42:14.993 INFO:teuthology.orchestra.run.smithi170.stderr:bash: line 1: ntpq: command not found 2024-09-17T08:42:14.995 INFO:teuthology.orchestra.run.smithi118.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T08:42:14.996 INFO:teuthology.orchestra.run.smithi118.stdout:=============================================================================== 2024-09-17T08:42:14.996 INFO:teuthology.orchestra.run.smithi118.stdout:^* hv01.front.sepia.ceph.com 3 6 377 63 +517us[ +488us] +/- 43ms 2024-09-17T08:42:14.996 INFO:teuthology.orchestra.run.smithi118.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 0 -2139us[-2139us] +/- 50ms 2024-09-17T08:42:14.996 INFO:teuthology.orchestra.run.smithi118.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 0 +2936us[+2936us] +/- 84ms 2024-09-17T08:42:14.996 INFO:teuthology.orchestra.run.smithi118.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T08:42:14.997 INFO:teuthology.orchestra.run.smithi170.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T08:42:14.997 INFO:teuthology.orchestra.run.smithi170.stdout:=============================================================================== 2024-09-17T08:42:14.998 INFO:teuthology.orchestra.run.smithi170.stdout:^* hv01.front.sepia.ceph.com 3 6 377 128 +596us[ +565us] +/- 42ms 2024-09-17T08:42:14.998 INFO:teuthology.orchestra.run.smithi170.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 64 -2113us[-2113us] +/- 49ms 2024-09-17T08:42:14.998 INFO:teuthology.orchestra.run.smithi170.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 64 +3004us[+3004us] +/- 83ms 2024-09-17T08:42:14.998 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T08:42:14.998 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-17T08:42:15.055 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-17T08:42:15.056 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-17T08:42:15.123 DEBUG:teuthology.orchestra.run.smithi118:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T08:42:15.202 DEBUG:teuthology.orchestra.run.smithi170:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T08:42:15.276 DEBUG:teuthology.orchestra.run.smithi118:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T08:42:15.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:42:15.306 DEBUG:teuthology.orchestra.run.smithi170:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T08:42:15.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:42:15.334 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-17T08:42:15.343 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-17T08:42:15.352 INFO:teuthology.task.internal:Duration was 1334.770109 seconds 2024-09-17T08:42:15.352 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-17T08:42:15.360 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-17T08:42:15.361 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T08:42:15.363 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T08:42:15.400 INFO:teuthology.orchestra.run.smithi118.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T08:42:15.413 INFO:teuthology.orchestra.run.smithi170.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T08:42:15.689 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-17T08:42:15.690 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi118.front.sepia.ceph.com 2024-09-17T08:42:15.690 DEBUG:teuthology.orchestra.run.smithi118:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T08:42:15.710 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi170.front.sepia.ceph.com 2024-09-17T08:42:15.710 DEBUG:teuthology.orchestra.run.smithi170:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T08:42:15.761 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-17T08:42:15.761 DEBUG:teuthology.orchestra.run.smithi118:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T08:42:15.763 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T08:42:15.831 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-17T08:42:15.831 DEBUG:teuthology.orchestra.run.smithi118:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T08:42:15.833 DEBUG:teuthology.orchestra.run.smithi170:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T08:42:16.111 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-17T08:42:16.121 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-17T08:42:16.121 DEBUG:teuthology.orchestra.run.smithi118:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T08:42:16.146 DEBUG:teuthology.orchestra.run.smithi170:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T08:42:16.171 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-17T08:42:16.180 DEBUG:teuthology.orchestra.run.smithi118:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T08:42:16.189 DEBUG:teuthology.orchestra.run.smithi170:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T08:42:16.209 INFO:teuthology.orchestra.run.smithi118.stdout:kernel.core_pattern = core 2024-09-17T08:42:16.234 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = core 2024-09-17T08:42:16.255 DEBUG:teuthology.orchestra.run.smithi118:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T08:42:16.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:42:16.282 DEBUG:teuthology.orchestra.run.smithi170:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T08:42:16.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:42:16.309 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-17T08:42:16.319 INFO:teuthology.task.internal:Transferring archived files... 2024-09-17T08:42:16.320 DEBUG:teuthology.misc:Transferring archived files from smithi118:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908211/remote/smithi118 2024-09-17T08:42:16.320 DEBUG:teuthology.orchestra.run.smithi118:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T08:42:16.361 DEBUG:teuthology.misc:Transferring archived files from smithi170:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908211/remote/smithi170 2024-09-17T08:42:16.361 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T08:42:16.397 INFO:teuthology.task.internal:Removing archive directory... 2024-09-17T08:42:16.397 DEBUG:teuthology.orchestra.run.smithi118:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T08:42:16.400 DEBUG:teuthology.orchestra.run.smithi170:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T08:42:16.446 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-17T08:42:16.458 INFO:teuthology.task.internal:Not uploading archives. 2024-09-17T08:42:16.458 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-17T08:42:16.467 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-17T08:42:16.467 DEBUG:teuthology.orchestra.run.smithi118:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T08:42:16.469 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T08:42:16.482 INFO:teuthology.orchestra.run.smithi118.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 08:42 /home/ubuntu/cephtest 2024-09-17T08:42:16.500 INFO:teuthology.orchestra.run.smithi170.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 08:42 /home/ubuntu/cephtest 2024-09-17T08:42:16.502 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-17T08:42:16.511 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-17T08:42:16.533 INFO:teuthology.run:Summary data: description: orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rotate-keys} duration: 1334.7701091766357 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-17T08:42:16.533 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T08:42:16.615 INFO:teuthology.run:pass